You are on this website now. My personal site was initially some static pages hosted on GitHub pages. It worked, but I felt that I could host it somewhere else while also learning about how most websites are hosted nowadays, especially when most of them are not static.
The site is now an Nginx container running on a Kubernetes cluster in Linode, built and deployed by a GitHub Actions pipeline on every push to main. A Prometheus instance in the cluster scrapes both the cluster itself and my Proxmox home lab server, which sits behind Tailscale rather than being exposed to the internet. The page you are on queries Prometheus over its HTTP API and renders those numbers live — the results are on my Home Lab Server project page.
I had previously had my site deployed on GitHub pages, but going to a Kubernetes cluster on Linode was going to be something more challenging. I have previously used Google Cloud and Kubernetes in college, and I was able to deploy a sample application for a class. It had been a couple years since I had done that type of work, so I did several tutorials on setting up Kubernetes clusters on Youtube and Kubernetes documentation. I also used Amazon Web Services for my initial learning. Once I got the hang of deploying clusters and configuring them once again, I was ready to try deploying my website with Kubernetes.
I thought about using Amazon Web Services to deploy my website, as that is what I used to re-learn Kubernetes. However, I decided not to use AWS as it was too costly and I put it on Linode instead. To make sure my website was organized for deployment on Kubernetes, I moved around and renamed files to prepare them for packaging with Nginx, and then I tested my page locally on Docker. The only part that was not working on this setup was Prometheus, as I ran that before locally on another port. Once I finished Docker testing, I created a Kubernetes cluster in Linode.
Getting Prometheus working was definitely the trickiest part of this project, as this required the most moving parts. Starting off, I added Prometheus and Tailscale pods and namespaces to enable my cluster to grab metrics from my server also on the Tailscale network. I proceeded to deploy my site image to the cluster. By creating a scrape config file to scrape my Proxmox server's metrics and adding it to the Prometheus pod and creating an egress proxy to allow the website pod to reach the Proxmox server, I got my server's metrics to display on my website.
I didn't want to keep manually deploying my website, especially as I continuously push changes to fix problems. So I wrote a GitHub Actions workflow that builds the image, pushes it to GitHub Container Registry, and applies the manifests to the cluster. The job waits on kubectl rollout status, so if a new pod never passes its readiness probe the run fails loudly while the old pods keep serving. Without that gate a broken deploy would show that it is working, but the site would be broken. Certificates are issued automatically by cert-manager through a ClusterIssuer, so the site stays on HTTPS without me renewing anything manually.
In case my cluster went down, I wanted to make sure I could recreate it easily and with the previous settings I had set it up with initially. I used Terraform to create a configuration file that would create the cluster with the same settings as before, so if I ever needed to recreate it, I wouldn't have to manually configure on the Linode dashboard. I had set up my cluster without the use of Terraform, so I had to figure out how to get the configuration file to match the cluster I had already created. I used the Linode CLI to get the cluster's ID and then used the Terraform import command to import the cluster into my configuration file. After that, I was able to use Terraform to manage my cluster.
As I was figuring out how to have my Prometheus read my server from Kubernetes, I thought about concerns about having my server exposed to the internet. I had a hard time figuring out how to expose my metrics securely until I learned that Prometheus can scrape metrics off of a Tailscale node. However, setting this up was complicated and it required a lot of research.
I learned that I could do it by adding a sidecar to my Prometheus pods. Rather than doing that though, I edited my Tailscale configuration and added an ACL rule to allow my Kubernetes nodes to access the ports that Proxmox runs off of. It took me a while to get this working as I was new to Tailscale's access control model and getting the correct rules and tags on the correct instances, but I finally got the Tailscale pod on my cluster to work.
With Prometheus scraping successfully, the site itself still showed nothing. The cause was that the website pod had no route to a service that only exists on the tailnet, thus the pod and the tailnet were still two separate networks. Following Tailscale's documentation, I created an egress proxy service in the cluster that maps a normal Kubernetes service name to the tailnet address, so the site's requests resolve like any other in-cluster call. After that the metrics rendered.
My Proxmox server runs on a laptop at home, so it won't be online all the time. I had to decide what my website should do when the server it pulls metrics from is unreachable. Initially, I had my Kubernetes health checks confirm that the metrics were actually working with the liveness and readiness probes. However, Kubernetes restarts a container when its liveness probe fails, and pulls a pod out of the load balancer when its readiness probe fails. If either probe depended on Prometheus, my home lab going offline would fail both of my website pods and the entire site would go down with 503 errors.
There was another problem with that approach that I missed initially. My pipeline waits on kubectl rollout status, so a new pod only counts as available once its readiness probe passes. If readiness depended on my home lab, I wouldn't be able to deploy anything at all while the laptop was off. Instead, I kept both probes as simple endpoints in my Nginx config that return 200 as long as Nginx is serving, which is the only thing that pod is actually responsible for. I handled the failure in the frontend instead, so when my server is unreachable the metric cards show "n/a" and the page reports how many queries failed, while the rest of the site keeps working normally.
While setting up my Terraform configuration, I realized that my nodes had no firewall, given that they were assigned public IP addresses. The only way the public should be able to access my website is through the NodeBalancer. Initially, I put rules allowing only ports 80 and 443, but I realized that these were firewall rules being placed on the nodes, not the NodeBalancer. Had I done this, it would have kept anyone from reaching my website. I learned that the nodes have a private network that the NodeBalancer can reach, so I made the firewall rules to only allow traffic from the private network, and put a default-deny rule for other inbound traffic. I also allowed outbound traffic, since the nodes need to pull images from the container registry and reach my home lab's metrics using Prometheus.
In this project, I learned about how CI/CD pipelines work and how to get a website on the cloud through Kubernetes. Putting my home lab server's metrics online also taught me about how backend services reach the frontend securely. I learned about containers, container orchestration and deployment of a project. Every part of deploying the website with Kubernetes from GitHub Pages was a difficult learning experience, but the most difficult part for me was getting my Proxmox metrics securely. By making sure that the cluster and the home lab are on the Tailscale network, this prevents an attacker from making calls to the backend arbitrarily.
I also learned the difference between running services such as Prometheus and Grafana on a local machine versus on a cloud instance. It would have been much easier to just have the screenshot of my server metrics, but I wanted to learn how to have them displayed live. By taking the additional step, I learned how to use Tailscale to expose my server to my cloud instance securely and how to configure Kubernetes egresses and scrape configs.