(TL;DR at the end)
I have been rebuilding my lab over the past couple of months, to get finally rid of the last 2 windows server VMs (active directory) and rebuild everything with mostly open source tools.
I was just setting up my main OpenShift cluster when the whole gitops rabbithole finally showed its true depths - so after setting up argo and trying to implement the app-of-apps pattern I realised that my strategy for getting secrets into my cluster needed some more thought. I found “external secrets” to be the best bet for this, maybe adding some SOPS into the mix, where needed.
So now I found myself setting up a vault instance, which brings me here today. Vault and consul should be configured with SSL (as everything should tbh) and I’m not too sure where to get my certs.
I have a domain, that I only use for internal stuff, so I would be able to get valid SSL certs through certbot, but my DNS server is internal only, which brought me to the http-01 challenge. This needs a file on a webserver (under http://example.com/.well-known/acme-challenge/ ) which is easily doable for VMs.
It gets a bit more tricky for cert-manager and Kubernetes. Here, the above URL is provided through the ingress-controller, which is internal only. I could theoretically expose those endpoints to the internet, but I don’t want the extra overhead of configuring all of that. At that point, it’d be easier to just manually generate certs and add them to the endpoints.
On the other hand, I have a freeIPA server running - for DNS and LDAP - that could theoretically also act as a CA, so I could generate my own certs and import the root cert on my devices. That’s nice for desktops and laptops, but with mobile devices, this seems like a lot of tinkering, which - again - I’d like to avoid.
So ultimately the question is: How do you handle SSL certs and ACME in your homelab and all the devices that are connecting to it?
TL;DR: Reduilding lab and rethinking SSL. I have a valid domain, but DNS is internal only. Letsencrypt is not too easy because of internal DNS and cert-manager on kubernetes ingrsses that aren’t connected to the internet - DNS challenge wouldn’t work and HTTP challenge would probably be a lot of work. Running my own CA seems like a lot of hassle with mobile devices. Need advice.
After tinkering for a few days I think I finally found the best comnpromise for my setup.
I’m using my public DNS (at cloudflare) to deploy the DNS challenge txt entries, to then issue certs using cert-manager and certbot.
This is security wise not the best solution because all these DNS entries are public, so mapping my internal infrastructure would be somewhat possible, but since it’s a homelab I’m not too concerned about this attack vector.