When you're building a cloud application, the last thing you want is to have to stop while you get yourself some certificates to secure your servers. It costs time and money and wouldn't you rather just print your own certificates for your own use? I mean, you trust you don't you? Assuming you do, then you can likely make use of mkcert when building out that cloud app.
If you know mkcert, you may not think of it as something for a cloud application's backend. Mkcert was originally developed with another problem in mind, local development. When putting together a web application to run in development on a local machine, it is literally impossible to get a certificate for it from a certificate authority - everyone has a localhost.
You can try and make your own self-signed certificates to get around that problem. That turns into a bit of a red flag carnival when you use them, with applications complaining about not being able to trust the certificates you've signed as they can't be checked with anyone. As the developers of mkcert say, "Managing your own CA is the best solution, but usually involves arcane commands, specialized knowledge, and manual steps."
Why mkcert
That's why they came up with mkcert which does all that arcane stuff for you, from making a root CA certificate to generating locally trusted certificates. And more importantly, it does it without the strange world of OpenSSL commands and error messages which can quite literally reduce people to tears.
Mkcert can get it all up and running and it'll talk to your local trust store and make your generated certificates trustable. And there are no weird commands or errors. It's great, but what's this got to do with certificates in the cloud.
Well, what mkcert does is also enough to enable TLS for many applications and servers, so you can turn to mkcert for some fast certificate provisioning for your cloud apps.
Applying mkcert
Let's show you how it works. Recently at Fly, we put together an example of a Redis installation with TLS enabled. To make it easy to reproduce and secure, we used mkcert.
The first step is to install mkcert. You'll find the instructions for that in it's README.
Once it's installed you run mkcert -install and that's where the magic happens. It creates a functioning local certificate authority (CA) with no intervention and installs that CA locally and, if Firefox is around, into Firefox. The only part of that we're interested in is the fact that it has created a CA because it's from that CA that we can extract the CA root file. This is the certificate that other applications can use to verify any certificates that this mkcert installation will generate.
Run mkcert -CAROOT to get the file name for the CA root file. Copy it somewhere - or combine it all into one command:
mkdir -p certs
cp "$(mkcert -CAROOT)/rootCA.pem" certs/rootCA.pem
You now have what is essentially a master key tester for your certificates. We'll be copying this file around our application's infrastructure and... will you stop panicking at the back there. First of all, this is a certificate that's been signed using another key so you can't make new certificates with it, only verify them.
Secondly, your application's infrastructure should be working like a sealed box, with only specific and auditable entry paths. Beyond common base images, it should be composed of assets and container images that you control access to.
Copying the CA Root file around inside a closed environment should not be an issue, and if it is, well, you have bigger problems to be concerned with. If you have external access to the system it should be secured by real, verifiable certificates anyway.
Cutting Server Keys
Anyway, now we have a CA Root file we can use to verify keys, we can get around to cutting some keys.
The first key we need in our Redis TLS example is one for the server, which has a hostname of appkata-redistls.fly.dev. Mkcert will generate a key and certificate for the server
mkcert -key-file certs/redis-server.key -cert-file certs/redis-server.crt appkata-redistls.fly.dev
That's the certificates done, all we need to do now is configure Redis with those certificates. The certs directory we created should be copied into /etc/ before this is done (as part of the image building) and with that in place we add:
tls-port 7379
tls-cert-file /etc/certs/redis-server.crt
tls-key-file /etc/certs/redis-server.key
tls-ca-cert-file /etc/certs/rootCA.pem
tls-auth-clients no
To the redis.conf file. That's very Redis specific, but in more generic terms, you pass the cert, key, and ca-cert file to the application's TLS configuration; either through a config file or through command-line flags.
When the server is brought online, it'll use those certificates and that CA root as a basis for validating the certificate. Of course, as no connecting client will know how to verify the locally generated certificates, you'll have to give the client a copy of the rootCA.pem file so it can do that for itself. Something like this:
redis-cli -h appkata-redistls.fly.dev \
--tls \
--cacert certs/rootCA.pem
An incoming client, equipped with that file, will be able to connect over TLS and authenticate with a password if you've set one. But all this does is encrypt the connection.
Keys for Clients
We can tighten up security further if we ask the server to demand that anyone connecting has a valid client certificate. The first step in that is turning that requirement on. Over to the redis.conf file and let's flip the tls-auth-clients line to read:
tls-auth-clients yes
Now you can't connect to the server - with Redis, you can connect but you'll be disconnected when you try any command. What we need now are some client certificates, ones that say "This client is allowed to access this named server". Again mkcert makes it simple. Just add --client to the mkcert and it will make client certificates.
mkcert --client -key-file redis-client.key -cert-file redis-client.crt appkata-redistls.fly.dev
Now, you can go to a remote system, with these files, redis-client.key, redis-client.crt and the rootCA.crt file and use them to connect as application such as redis-cli:
redis-cli -h redis-example.fly.dev \
--cert redis-client.crt \
--key redis-client.key \
--tls \
--cacert certs/rootCA.pem \
Summary
We've gone over the processes involved in creating keys and certificates for servers and client and setting up a Redis server, without and with client authentication. It's important to know that these mkcert certificates we've generated have a limited life span, at which point you'll have to manually generate and install them again. So while they are great for your development processes, you'll want something more automated when it comes to production. Which we'll come to in a future article.
Top comments (2)
Nicely put walkthrough! How do you use it beyond local development? Hardening connections between services internally? (Say web app to Redis). I imagine it's relevant for scenarios where you manage most of the infrastructure yourself. If using cloud-provided services they usually implement TLS connections natively. Curious to know your use cases and ways you tackle certificate upgrades in an automated fashion :)
Cloud provided services are already pointing out into the internet, so really do have to have an externally verifiable certificate. At Fly we globally run vms using Firecracker, so users can construct their application infrastructure which mostly talks to other nodes in the infrastructure wherever they need it. For externally facing sites, Fly automatically does TLS for development sites, and we make it easy to attach custom domain names with TLS.
By using locally generated certificates it's quick to test out that, and then when heading to production either sort out automation (as I said coming soon, along with more cert handling tricks) or get real certificates which have longer life times, but externally validate.