This post is going to run through how I created nebloc.com some of the challenges I faced, and areas that I would like to improve.

It is worth noting that the site should probably not be copied. I chose the technology stack and solution on the ‘requirements’ of what I wanted to learn, not on the best performing, ease of maintenance (it’s quite the opposite), etc.

Overview#

Napkin Sketch
Napkin Sketch

This is a basic napkin drawing of my site. The main services used are Cloudflare, Hetzner, Azure Devops, and Google Cloud Platform (GCP).

Cloudflare are providing DNS for ’nebloc.com’ and free CDN, which for a site that mainly hosts images is fantastic. Hetzner are my cloud provider of choice where I am ‘renting’ (is that the right word? 🤔) 3 nodes of 2 VCPU and 4GB of RAM each (~€15). This of course is completely overkill, much like the rest of the architecture!

The 3 nodes were provisioned with Terraform, connected on a private network with Wireguard, and Kubernetes installed across them with Weave Net, etc(d).

Azure Devops provides a free plan for teams of up to 5, which suites me perfectly, so I am using them for Git hosting and pipelines. I Have started to use their Kanban/Scrum board features as well but not ready to give an opinion.

The web application’s docker images are stored in Google Container Registry (private and fairly cheap as just pay for storage and egress). The photos to display are served from a Google Storage Bucket, sorted in to directories for each ‘Collection’. Collection details (description, location, dates, etc.) are stored in Google Firestore.

History#

I naturally gravitate to GCP as a cloud provider as it is the one I have had the most experience with, and find the easiest interface / level of abstraction. Therefore the website started as a Google App Engine project.

After a while the not-so dynamic nature of the site was annoying me and I had some time to kill, and a want to learn Kubernetes (k8s)! BUT I don’t have £30pm to spend on a cluster that no one really visits that often, and is completely overkill for what I should be doing.

My search lead me to this fantastic repo: hobby-kube. Not only does it have a guide that was fantastic for learning bits of Kubernetes but it also provides terraform provisioning for 3 cloud providers that are relatively cheap; Scaleaway, Hetzner and Digital Ocean (DO). Although I like DO, they were the least cost effective of the three (although their managed k8s offering is a great deal!), and Hetzner offered great ‘bang for your buck’.

Once I got k8s up and running I installed tools such as Cert-Manager for SSL, Nginx Ingress controller for, well, ingress, and a few other tools for monitoring (Loki + Prometheus).

Eventually I started to use Cloudflare for DNS and CDN as they have a generous free tier. This also gave me SSL management at their level so I removed Cert-Manager from my cluster, and implemented a force-ssl rule on the ingress to ensure users are coming through HTTPS.

At work we have taken the decision to use Azure as our preferred cloud service, and as such I got the opportunity to have some training on Azure Devops. Originally I was trying to do CI/CD with GOCD running inside my cluster with Docker-in-Docker agents. This worked but needed persistent storage, which is a cost (albeit a small one), and was fragile in deploying resulting builds to the cluster - see their example. Again Azure Devops provides a free plan and was surprisingly easy to set up deployments with Helm to my cluster using their Service Connections.

This then prompted me to change my Git hosting. It started on Github (free private repos for students) but when my Student plan ran out I switched to Gitlab (which I still use for most projects). Azure Devops only supports YAML pipelines for certain repo providers so I switched the website (monorepo) to host it in Azure Devops.

In its current state the pipelines build the containers on changes to develop / master branches. Depending on the branch it will deploy to k8s with different ingress settings, i.e. master will deploy with host: nebloc.com and nginx.ingress.kubernetes.io/from-to-www-redirect: "true".

Wrap Up#

As it stands I am quite proud of the current state of everything. The pipelines especially are a massive boost to the speed of change I can make to the site. They removed a lot of the headache of finishing a feature, wanting to share it, and then banging my head against a wall until 4 A.M. in the morning taking the site up and down!

I am hoping to focus more on the ‘admin’ side of things and write some CLI tools to help manage new collection releases, by automating the compression / upload of images, add the relevant data to Firestore, etc. I have also started investigating Sendgrid for creating a mailing list and a contact me section to the site.

In future posts I will go in to more detail on what the images I am building are, what frameworks I have chosen, and what users are actually interacting with.