MicroK8s - Setup an nginx ingress with cert-manager
Sorry, what? microK8s, Ingress, cert-manager? 🤔
Disclaimer
I won’t explain every detail in this article, it’s mainly for my own note taking. If you read this article, I suppose you already know how to setup your DNS, setup some basic HTTP routes, deploy some deployments, services on a K8s cluster.
Why I use Kubernetes
Kubernetes (K8s) is a container orchestration tool, many cloud providers like Google, Amazon, DigitalOcean have their own proprietary implementation of Kubernetes and allow you to setup a Kubernetes cluster with a few clicks. A Kubernetes cluster offers many advantages, some of them target big companies or critical apps with High Availability, Replication, and so on. For my own projects, I’m more interested in using a tool that allows me to automatically orchestrate my own containers and automate their deployment.
Why I decided to try MicroK8s
I tried the cloud solutions, and loved them. One day, I had to update my cluster with the update process of DigitalOcean (automatic) and… BOOM 💥 all of my sites went down, the load balancer was down. I had to setup the load balancer again and I wasn’t happy with the idea of paying for the High Availability stuff, which wasn’t highly available AT ALL.
I found an article about MicroK8s: a lightweight K8s implementation that you can install almost anywhere. Ok, it means you need to install it and you don’t have the master node that comes for free. BUT, you can install it anywhere, and to be honest, I don’t think I need all of the DigitalOcean High Availability promises for my small apps.
So, I decided to try managing my own kubernetes cluster on a Vultr VPS instance. Their servers have better performances for the same price and people say their support is great (compared to the bad support at DO).
Ingress and cert-manager
An ingress controller lets you route your URLs to different services in your kubernetes cluster. It generally sits behind a Load Balancer (which sometimes manages your SSL certificates). In this article, I’ll install MicroK8s with some services and route some urls to them. I’ll use Cert-manager to manage the SSL certificate (with let’s encrypt).
Setting up MicroK8s
Install MicroK8s with snap
The instructions are available from the official website: https://microk8s.io/#get-started
I’ll use the guide for Ubuntu as I’m installing it on fresh new DigitalOcean droplet.
1 | sudo snap install microk8s --classic --channel=1.18/stable |
Optional: get remote access via a SSH tunnel
1 | # Export the custom config |
1 | # Set a variable to your server IP |
Optional (2): get remote access without a SSH tunnel
⚠️ This method exposes the 16443
API port to the web, you might not want to expose it for production use cases ⚠️
1 | # Export the custom config |
1 | # Set a variable to your server IP |
Test deployment
For the purpose of this article, I’ll deploy a basic nginx deployment and expose it.
I’ll also create another deployment/service called hello-deployment with the same method.
1 | kubectl apply -f https://k8s.io/examples/application/deployment.yaml |
Then type kubectl get all
should display the following status:
1 | NAME READY STATUS RESTARTS AGE |
Ingress
Installing an nginx ingress controller with microK8s is easy, just type the following command to enable the corresponding plugin:
1 | microk8s.enable ingress |
Then type kubectl get all --namespace ingress
should return the following status, with a new daemonset and pod for the ingress controller.
1 | NAME READY STATUS RESTARTS AGE |
Now, let’s create a basic ingress definition, this will route ilightshow.app/ to the nginx-deployment, and route ilightshow.app/hello to the hello-deployment. Please note that you need to setup your DNS records for your domain name.
1 | apiVersion: extensions/v1beta1 |
Cert-manager
Setup your DNS
Don’t forget to setup the DNS record for your domain name, it needs to point to your server, otherwise, you won’t be able to reach the deployments by typing the URL (for me: http://ilightshow.app). AND, the Let’s Encrypt servers won’t be able to reach the cert-manager challenge (If you don’t understand this, please read this article).
Install cert-manager
First, run microk8s.inspect
on the server, make sure you don’t have any warning. For instance, I had the following warning:
WARNING: IPtables FORWARD policy is DROP. Consider enabling traffic forwarding with: sudo iptables -P FORWARD ACCEPT
This part is based from this tutorial.
1 | # Create a new namespace for the cert-manager |
Deploy a staging and prod issuer
Create the following files and apply them with kubectl apply -f
1 | apiVersion: cert-manager.io/v1alpha2 |
1 | apiVersion: cert-manager.io/v1alpha2 |
Setup the ingress to work with cert-manager
Just add the following annotations, it will ask the cluster-issuer to issue a certificate.
1 | apiVersion: extensions/v1beta1 |
Then, check that the certificate is ready with kubectl get certificate
. If everything worked fine, you can change the issuer to the prod
issuer.