DEV Community

Cover image for TLS and mTLS for Infra Services: A Production Deep Dive for Solo Devs and Small Startups
instanceofGod
instanceofGod

Posted on

TLS and mTLS for Infra Services: A Production Deep Dive for Solo Devs and Small Startups

Why this exists

This started as a cost problem, not a security paper.

I was running a small production setup across two servers: a dedicated 1cpu/1GB RAM server running Redis, and a second OCI server — 2ocpu/12GB RAM — hosting a Postgres database alongside several NestJS app containers. Two servers, many services, and every one of them talking in plaintext by default. The app containers reached Redis over the private network; they reached Postgres on the same host they lived on. But "private network" and "same host" aren't the same thing as encrypted — I'd locked things down with iptables (as oracle compute manages firewall with iptables by default), restricting which hosts could even reach the Redis and Postgres ports, and routed the app-to-Redis traffic over a private IP instead of a public one. That's necessary hygiene, but it's not TLS. Anyone with a foothold on that private network, or one misconfigured firewall rule, would see everything moving in the clear.

So the real question became: how do I get Redis, Postgres, and my app containers actually encrypting traffic between them — without turning a two-server side project into one that also needs its own PKI team?

That's the situation most solo devs and small startups are actually in. You're setting up and managing Redis, Postgres, and maybe Kafka, and every one of them defaults to plaintext traffic between services. The "proper" answers people point you to — a managed PKI service, HashiCorp Vault, cert-manager on Kubernetes, a paid CA — all assume you already have infrastructure to run infrastructure on. That's not where you are on day one.

This guide is what came out of solving that for my own setup: two servers, three services, openssl, and no budget line for security tooling.

The good news: TLS between your own services doesn't need any of that. openssl and a private Certificate Authority (CA) you run yourself gets you encrypted, authenticated traffic between Redis, Postgres, Kafka, or any custom service — for free, with tools already on your machine, and without a dependency on anyone else's uptime. The tradeoff is that you now own certificate lifecycle management: rotation, expiry, and key custody. This guide covers both halves — how to generate and understand every file involved, and how to run it in a way that won't fall over in production or bite you six months from now when a cert silently expires.


Part 1 — TLS and mTLS, the mental model

Plain TLS: only the server proves who it is. The server holds a private key and a certificate; the client checks that certificate against a CA it trusts. Encrypted traffic, one-directional trust. This is what almost every HTTPS connection on the internet does.

Mutual TLS (mTLS): both sides prove who they are. The client also holds a private key and certificate, and the server verifies it before accepting the connection. This is the right model for internal service-to-service traffic — it's not just "is this connection encrypted," it's "is this actually my app server talking to my database, and not something else on the network."

The rule that matters most in everything below: private keys never leave the machine that generates them. Only certificates (public) and CSRs (also public — a request, not a secret) ever cross the wire. If you find yourself scp-ing a .key file between machines, stop and generate it in place instead.


Part 2 — Set up your Certificate Authority

Everything downstream depends on one CA you control. This is the only step where the "signed by itself" special case applies.

Directory layout

sudo mkdir -p /etc/pki/tls
sudo chmod 700 /etc/pki/tls
cd /etc/pki/tls
Enter fullscreen mode Exit fullscreen mode
/etc/pki/tls/
├── ca.key       # CA private key — the crown jewels, guard this
├── ca.crt       # CA certificate — distributed to every service and client
├── server.key   # per-service private key
├── server.crt   # per-service certificate, signed by ca.key
├── server.csr   # per-service signing request
├── client.key   # per-client private key (mTLS only)
├── client.crt   # per-client certificate (mTLS only)
└── client.csr   # per-client signing request (mTLS only)
Enter fullscreen mode Exit fullscreen mode

Generate the CA private key

openssl genrsa -out ca.key 4096
Enter fullscreen mode Exit fullscreen mode
  • genrsa — generate an RSA private key.
  • -out ca.key — write it to this file.
  • 4096 — key length in bits. 2048 is the accepted minimum; 4096 is the conservative choice specifically for a CA, since it's your root of trust and gets used to sign everything else.

Output: ca.key — a raw private key. Not yet a certificate.

Turn the CA key into a self-signed CA certificate

openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 \
  -out ca.crt -subj "/CN=Internal-CA"
Enter fullscreen mode Exit fullscreen mode
  • req — the certificate-request subcommand; combined with -x509 it emits a full certificate instead of a request.
  • -x509 — self-sign directly. This is what makes it a root CA — signed by itself, trusted by nothing but its own authority.
  • -new — generate fresh material rather than reading an existing request.
  • -nodes — "no DES": don't password-protect the private key. Omit this if you want ca.key passphrase-protected (more secure, but you'll be prompted every time you sign something with it — worth it once you're not the only one operating this).
  • -key ca.key — the key this cert is built from.
  • -sha256 — signature hash algorithm.
  • -days 3650 — 10-year validity. Long, deliberately — this is not something you want to rotate casually, since rotating it means re-issuing and redistributing every downstream cert.
  • -subj "/CN=Internal-CA" — sets identity non-interactively. CN is just a label for a CA, not a hostname.

Output: ca.crt — the certificate every service and client will be told to trust.

Guard ca.key. Anyone holding it can mint certificates your infrastructure will trust unconditionally. At minimum: chmod 600 ca.key, restrict which accounts can read the directory, and keep an encrypted offline backup. As you grow past "just me," this is the first thing to move into a secrets manager rather than a bare file on disk.


Part 3 — Issue a server certificate (Redis, Postgres, Kafka, or anything else)

This pattern is identical regardless of which service is consuming the cert — generate a key, generate a CSR, have the CA sign it.

Generate the service's private key

openssl genrsa -out server.key 2048
chmod 600 server.key
Enter fullscreen mode Exit fullscreen mode

2048 bits is fine here — service keys get rotated far more often than the CA key, so they don't need the extra margin.

Create the certificate signing request (CSR)

openssl req -new -key server.key -out server.csr \
  -subj "/CN=redis.internal.example.com"
Enter fullscreen mode Exit fullscreen mode
  • -new (no -x509) — this produces a request to be signed by the CA, not a self-signed cert.
  • -subj "/CN=..." — set this to the hostname (or IP) clients will actually connect to:
  CN=redis.internal.example.com
  CN=postgres.internal.example.com
  CN=kafka-broker-1.internal.example.com
  CN=10.0.1.140
Enter fullscreen mode Exit fullscreen mode

A CSR bundles the public key and identity info, signed by the service's own key to prove possession of it — but it carries no trust until the CA signs it.

Sign it with your CA

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out server.crt -days 825 -sha256 \
  -extfile <(printf "subjectAltName=DNS:redis.internal.example.com,IP:10.0.1.140")
Enter fullscreen mode Exit fullscreen mode
  • x509 -req — read the CSR, output a signed certificate.
  • -CA ca.crt -CAkey ca.key — the CA doing the signing. This is the step that makes server.crt trusted by anything that trusts ca.crt.
  • -CAcreateserial — creates ca.srl, a serial-number tracking file so every cert this CA signs gets a unique serial. Required bookkeeping, not optional.
  • -days 825 — ~2 years, near the maximum modern TLS clients/browsers will accept for a server cert. Internal service traffic is more lenient, but there's little reason to go longer.
  • -extfile <(printf "subjectAltName=...")do not skip this. Modern TLS validates the connection against the certificate's SAN (Subject Alternative Name) field, not CN. Without SAN, many clients (including recent Postgres and Redis clients) will reject the certificate even though CN looks correct.

Verify what you built

openssl x509 -in server.crt -text -noout        # inspect the cert
openssl verify -CAfile ca.crt server.crt         # confirm the chain is valid
Enter fullscreen mode Exit fullscreen mode

Expected: server.crt: OK.

File permissions before anything goes near production

sudo chown redis:redis /etc/pki/tls/server.key /etc/pki/tls/server.crt
sudo chmod 600 /etc/pki/tls/server.key
sudo chmod 644 /etc/pki/tls/server.crt /etc/pki/tls/ca.crt
Enter fullscreen mode Exit fullscreen mode

Swap redis for whichever account runs the service (postgres, kafka, appuser, etc). Postgres in particular will refuse to start entirely if server.key is group- or world-readable — it's not a soft warning.


Part 4 — Configuring real services

The certificate files are identical in shape across services; what differs is the config syntax each one expects.

Redis

redis.conf:

port 0
tls-port 6379
tls-cert-file /etc/pki/tls/server.crt
tls-key-file  /etc/pki/tls/server.key
tls-ca-cert-file /etc/pki/tls/ca.crt
tls-protocols "TLSv1.2 TLSv1.3"

# Require a client cert (mTLS) — omit or set "no" for one-way TLS
tls-auth-clients yes

# If you run replication or cluster mode, TLS has to be turned on for those links separately
tls-replication yes
tls-cluster yes
Enter fullscreen mode Exit fullscreen mode

Setting port 0 disables the plaintext listener entirely — worth doing once TLS is confirmed working, otherwise Redis will happily accept unencrypted connections on the old port alongside the TLS one.

Connect and test:

redis-cli --tls \
  --cert client.crt --key client.key --cacert ca.crt \
  -h redis.internal.example.com -p 6379 ping
Enter fullscreen mode Exit fullscreen mode

Postgres

postgresql.conf:

ssl = on
ssl_cert_file = '/etc/pki/tls/server.crt'
ssl_key_file  = '/etc/pki/tls/server.key'
ssl_ca_file   = '/etc/pki/tls/ca.crt'
Enter fullscreen mode Exit fullscreen mode

pg_hba.conf — this is where you decide whether client certs are actually required (mTLS) or merely accepted:

# Require a valid client cert signed by your CA, on top of TLS
hostssl  all  all  0.0.0.0/0  cert

# Or: TLS required, but authenticate with a password (one-way TLS)
# hostssl all all 0.0.0.0/0 scram-sha-256
Enter fullscreen mode Exit fullscreen mode

Client side — either via connection string or ~/.postgresql/{postgresql.crt,postgresql.key}:

psql "host=postgres.internal.example.com dbname=app user=appuser \
  sslmode=verify-full \
  sslrootcert=/etc/pki/tls/ca.crt \
  sslcert=/etc/pki/tls/client.crt \
  sslkey=/etc/pki/tls/client.key"
Enter fullscreen mode Exit fullscreen mode

sslmode=verify-full is the one you want in production — it validates both the chain and that the hostname matches the cert's SAN. verify-ca skips the hostname check; require skips both and only gets you encryption, no authentication of the server at all.

Kafka

Kafka's brokers and clients expect Java KeyStore format (JKS or PKCS12), not raw PEM files, so there's an extra conversion step your Redis/Postgres setup doesn't need.

# Bundle the server cert + key + chain into a PKCS12 keystore
openssl pkcs12 -export \
  -in server.crt -inkey server.key -chain -CAfile ca.crt \
  -out server.p12 -name kafka -password pass:changeit

# Convert to a JKS keystore
keytool -importkeystore \
  -destkeystore kafka.keystore.jks -deststorepass changeit \
  -srckeystore server.p12 -srcstoretype PKCS12 -srcstorepass changeit \
  -alias kafka

# Build a truststore containing your CA, so brokers/clients know who to trust
keytool -import -trustcacerts -noprompt \
  -alias CARoot -file ca.crt \
  -keystore kafka.truststore.jks -storepass changeit
Enter fullscreen mode Exit fullscreen mode

Broker server.properties:

listeners=SSL://kafka-broker-1.internal.example.com:9093

ssl.keystore.location=/etc/pki/tls/kafka.keystore.jks
ssl.keystore.password=changeit
ssl.key.password=changeit
ssl.truststore.location=/etc/pki/tls/kafka.truststore.jks
ssl.truststore.password=changeit

# mTLS: require a valid client cert
ssl.client.auth=required
Enter fullscreen mode Exit fullscreen mode

Client client.properties:

security.protocol=SSL
ssl.truststore.location=/etc/pki/tls/client.truststore.jks
ssl.truststore.password=changeit
ssl.keystore.location=/etc/pki/tls/client.keystore.jks
ssl.keystore.password=changeit
ssl.key.password=changeit
Enter fullscreen mode Exit fullscreen mode

Test:

kafka-console-producer.sh \
  --broker-list kafka-broker-1.internal.example.com:9093 \
  --topic test --producer.config client.properties
Enter fullscreen mode Exit fullscreen mode

Any other service (generic pattern)

Most services that support TLS want the same three inputs — adapt the directive names to whatever the service's docs call them:

Concept File Typical directive names
Service certificate server.crt tls-cert-file, ssl_cert_file, ssl.keystore
Service private key server.key tls-key-file, ssl_key_file, ssl.key.password
Trusted CA ca.crt tls-ca-cert-file, ssl_ca_file, ssl.truststore

Part 5 — Issuing client certificates (mTLS)

Same three-step pattern as the server cert, run on the client instead.

# on the client
openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr -subj "/CN=app-service-1"
Enter fullscreen mode Exit fullscreen mode

CN here is just an identifying label — it doesn't need to be a resolvable hostname the way a server cert's does, since nothing connects to the client by name. Make it something you can trace back to a specific service or team if you ever need to revoke it.

# on the CA host — sign the CSR, don't generate it there
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out client.crt -days 825 -sha256
Enter fullscreen mode Exit fullscreen mode

No SAN extension needed — nothing validates a hostname against a client cert, only identity/authorization.

How "presenting" a client cert actually works, mechanically: client.key never leaves the client as a file. During the handshake, the client uses client.key locally to sign part of the handshake data and sends only the signature to the server — never the key. The server verifies that signature against the public key embedded in client.crt. That's how the client proves possession of the private key without the key ever crossing the network. The same applies to server.key on the service side.


Part 6 — Testing connectivity generically

Before trusting any service-specific client, confirm the handshake itself works:

openssl s_client -connect redis.internal.example.com:6379 -CAfile ca.crt
Enter fullscreen mode Exit fullscreen mode

A successful one-way handshake ends with Verify return code: 0 (ok). For mTLS, add -cert client.crt -key client.key to present a client identity too — if the server is enforcing client auth, the connection will fail without them.


Part 7 — Certificate renewal, before it renews you

Certificates expire, and an expired cert in production is an outage, not a warning. Treat renewal as part of the setup, not a future problem.

Check expiry:

openssl x509 -in server.crt -noout -dates
Enter fullscreen mode Exit fullscreen mode
notBefore=Jul 24 2026
notAfter=Oct 27 2028
Enter fullscreen mode Exit fullscreen mode

Minimum viable renewal process for a solo dev / small team:

  1. A cron job or systemd timer that runs weekly, checks notAfter on every cert in /etc/pki/tls, and alerts (email, Slack webhook, whatever you already check) when anything is inside 30 days of expiry.
  2. Re-run the CSR + signing steps above for the expiring cert — same key or a fresh one, your call; rotating the key too is marginally better hygiene.
  3. Replace the file, restart or reload the service (most services support a config reload without a full restart for cert changes — check the specific service's docs).

This is intentionally manual. It's also completely adequate for a handful of services. The point where it stops being adequate is covered next.


Part 8 — When to stop doing this by hand

The manual openssl workflow above is free, fully understood, and has zero external dependencies — which is exactly why it's the right starting point for a solo dev or early-stage startup. It stops being the right tool once any of these become true:

  • You have more than a handful of services/certs, and tracking expiry by hand becomes error-prone.
  • You have more than one person who needs to issue certs, and sharing ca.key around by hand is no longer acceptable.
  • You need short-lived certs (hours/days instead of years) as a security posture, which requires automation to be usable at all.
  • You're running Kubernetes, where cert-manager with a self-hosted step-ca or Vault PKI backend gives you the same private-CA model but with automatic issuance and renewal wired into your deployment pipeline.

At that point, the migration path is straightforward because the underlying model doesn't change — you're still running your own CA, still trusting ca.crt the same way, still using SAN-bearing server certs and (optionally) client certs. Tools like step-ca or Vault PKI secrets engine just take over the CSR-generation-and-signing loop and add automatic rotation, audit logging, and API-driven issuance. You're not throwing away what you built — you're automating the same three-step pattern (key → CSR → sign) that every cert in this guide already follows.

Paying for a managed PKI or a service like Let's Encrypt-adjacent tooling starts to make sense specifically when the coordination cost of manual rotation exceeds its price — not before. For two servers and a handful of services, that point is further out than most guides imply.


Production-readiness checklist

  • [ ] ca.key permissions are 600, owned by a dedicated account, and backed up encrypted somewhere off the host that holds it.
  • [ ] Every server.crt includes a subjectAltName matching the exact hostname/IP clients connect to.
  • [ ] Private keys (server.key, client.key) are never committed to git, never sent over chat/email, and never copied between machines — regenerate in place instead.
  • [ ] tls-min-version / ssl_min_protocol_version equivalents are pinned to TLS 1.2 minimum; prefer 1.3 where the service supports it.
  • [ ] Certificate expiry is monitored on a schedule, with alerting well before the 30-day mark.
  • [ ] File permissions on keys are 600, certs are 644, and both are owned by the service account, not root or your personal user.
  • [ ] mTLS (client certs) is enabled on anything that isn't meant to be reachable by "anyone who can route to the port" — which, on internal infra, is almost everything.
  • [ ] You have a documented (even if manual) process for what happens when ca.key is suspected compromised — because "regenerate everything" is a much worse day if you're figuring out the steps for the first time under pressure.

Quick reference

# --- CA (once) ---
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 \
  -out ca.crt -subj "/CN=Internal-CA"

# --- Server cert (per service) ---
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -subj "/CN=service.internal.example.com"
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out server.crt -days 825 -sha256 \
  -extfile <(printf "subjectAltName=DNS:service.internal.example.com")

# --- Client cert (per client, mTLS only) ---
openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr -subj "/CN=client-name"
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out client.crt -days 825 -sha256

# --- Verify ---
openssl verify -CAfile ca.crt server.crt
openssl s_client -connect host:port -CAfile ca.crt

# --- Check expiry ---
openssl x509 -in server.crt -noout -dates
Enter fullscreen mode Exit fullscreen mode

Every certificate in this guide follows the same three-step shape: generate a key → generate a CSR → have the CA sign it into a certificate. The CA itself is the only exception, since it signs itself. Once that pattern is internalized, adding TLS to the next service — whatever it is — is the same five commands with a different hostname in the -subj field.

Top comments (0)