Every service I self-host sits behind one front door: Authelia,
doing single sign-on and 2FA for the lot — the dashboard, the password vault, the
VPN control plane, a couple of internal tools, the monitoring. One login, one
place to enforce a policy, one place to revoke access. When people say
"security first" about a homelab, this is usually the picture they have in mind.
I want to tell you the honest version of that picture, because the tidy diagram
and the reality are different things, and the gap is where all the learning is.
The short version:
- The architecture is genuinely good and I'd build it the same way again.
- I ran it for a while with the example config's placeholder secrets still in it, on an internet-facing portal. That's the £0 mistake in the title, and £0 is exactly what it would have cost an attacker.
- The two failures that cost me the most time weren't Authelia's fault at all — they were the edge in front of it and the DHCP lease underneath it. Which is the real lesson: with SSO, you don't secure a service, you secure a chain, and the chain breaks at the joints.
None of this is a "look how secure I am" post. It's the opposite. Here's what
broke.
The shape of it
Two integration patterns cover everything I run, and it's worth knowing both
exist because people reach for the wrong one constantly.
1. Reverse-proxy forward-auth — for apps that have no real auth of their own,
or whose auth I don't trust. The proxy asks Authelia "is this person allowed?"
before the request ever reaches the app. The app never sees an unauthenticated
request. This is how I gate the internal, LAN-only tools: they were written
assuming something in front would handle identity, and Authelia is that
something.
2. Native OIDC — for apps that speak OpenID Connect properly (the password
vault, the VPN dashboard, my main control panel). Here Authelia is a real
identity provider: the app does the OAuth dance, gets an ID token, and maps the
user from the sub claim. Cleaner, because the app knows who you are, not just
that you're allowed.
The rule I'd tattoo on a new self-hoster: forward-auth answers "may this
request through?"; OIDC answers "who is this?" If the app needs to know the
user — per-user data, roles, an audit trail — it wants OIDC. If you just need a
wall, forward-auth is less to get wrong.
In front of all of it: a reverse proxy terminating TLS with real certificates,
and — this part matters later — a single machine running that proxy, on a single
IP that every hostname resolves to.
What worked
Credit where it's due, because plenty did.
- One policy, everywhere. Turning on 2FA is one change in one place and every service inherits it. Revoking a person is one edit. That centralisation is the entire value proposition and it delivers.
- Defense in depth actually composed. Authelia writes a log line for every auth attempt. A separate intrusion-detection tool tails that log, and when it sees a run of failures it pushes the offending IP up to my CDN's edge to be blocked before it reaches my network at all. Authelia detects; the edge blocks. Neither knows about the other beyond a log file and an API, and that loose coupling is why it's held.
- PKCE public clients for the browser apps. No client secret sitting in a single-page app that can't keep one. The right modern default, and Authelia supports it cleanly.
If I'd stopped writing here it'd be a smug little post. Keep going.
Didn't work #1: I shipped the example secrets
This is the one that matters, so it goes first.
Authelia's example configuration ships with placeholder secrets — literal
strings like:
session:
secret: 'unsecure_session_secret_change_it'
storage:
encryption_key: 'a_very_long_secret_encryption_key_at_least_32_chars'
The names are practically a note left by the maintainers: change these. I
had changed some of them. I had not changed these two. And the portal was
internet-facing, behind a CDN, doing real SSO for real services.
The session secret signs the cookie that says "this person is logged in." The
encryption key protects secrets at rest in the database. Both were the value
printed in a public example that every scraper and every attacker already has.
Anyone who knew I ran Authelia — and the login page announces that loudly —
could have forged a session or decrypted the store. Cost to them: zero. It's the
default credential problem wearing a more sophisticated outfit, and I walked
straight into it because I'd mostly hardened the config and assumed "mostly"
was "done."
The fix was a careful rotation, and the procedure is worth writing down
because rotating an encryption key is not the same as swapping a password —
you have to re-encrypt the existing data:
- Stop the container. Do database work with no writers, never on a live DB.
- Back up the database (and its write-ahead-log files).
- Run the key-change command before editing the config — it reads the old key from the still-unedited file to decrypt, then re-encrypts with the new one. Edit the YAML first and you've thrown away the only copy of the old key.
- Replace the placeholders. Then assert no placeholder text survives anywhere.
- Run the verification command → expect
SUCCESS. - Start, and health-check.
Two things I now do every time, and recommend to anyone:
- Verify by fingerprint, never by printing. I confirmed the config, the secret store, and the running container all agreed by comparing SHA-256 hashes, not by echoing the values to a terminal that logs to who-knows-where.
-
The one-off tool container fights you. Authelia's entrypoint does a
[ "$1" != "--config" ] && exec "$@", sodocker run ... authelia/authelia storage encryption change-key ...tries to execute a binary calledstorageand dies. You have to name the binary:docker run ... authelia/authelia authelia storage .... Twenty minutes of "why won't this run" before I read the entrypoint.
The lesson isn't "change your secrets" — you know that. It's that "mostly
configured" is the most dangerous state a security tool can be in, because it
looks and behaves exactly like "fully configured" right up until it doesn't.
Didn't work #2: the edge 403'd the one call that matters
I stood up OIDC login for my main dashboard. Entered my password, did 2FA,
got redirected back… and hit:
Provider unreachable: JWKS lookup failed: HTTP Error 403: Forbidden
Login had succeeded. Authentication was fine. It died at the very last step:
the app fetching Authelia's public signing keys (the JWKS) to verify the
token it had just been handed.
The cause took embarrassingly long to find. My Authelia is behind a CDN with
bot-fight protections on, and those protections 403 any request with a
Python-urllib User-Agent at the edge. Here's the trap: the app's OIDC library
made its discovery and token calls with a modern HTTP client (allowed UA, sailed
through) — but the token-verification step used a different, lower-level
library that fell back to bare urllib. Same server, same URL, two libraries,
two User-Agents, one blocked.
I proved it with a one-liner: the exact JWKS URL returned 200 to a browser
User-Agent and 403 to Python-urllib. The fix was to pass an explicit
browser-like User-Agent header to the key-fetching client. One header.
The lesson here is the one that generalises past Authelia entirely: when you
put a security layer in front of a security layer, they will disagree about
who's allowed, and the disagreement won't look like a permissions error. It
looked like an outage. "Mostly worked, failed on the last call, with a message
that blamed the wrong component" — that's the signature of a defense-in-depth
seam, and now it's the first thing I check.
(Worth adding: this fix is a local patch that reverts if I reinstall the app.
So I keep a break-glass login — a normal password provider, disabled, but one
edit away — precisely so that if the OIDC path breaks again after an update, I'm
not locked out of my own front door by my own security. Plan the way back in
before you shut the other doors.)
Didn't work #3: "Authelia down (×87)" — and it was never Authelia
My monitoring paged me: DOWN: Authelia, 87 times in a row, plus the node
it runs on. Authelia was answering 200 on its own port the entire time.
Both my hosts had rebooted after a power event. The one running the reverse
proxy came back up and DHCP handed it a different IP than the one it always
had — because, it turned out, I'd never made that IP a proper reservation. It had
just been a sticky lease for months, and I'd mistaken "it's always been .120"
for "it's pinned to .120."
That one machine runs the estate's only reverse proxy, and every hostname —
the auth portal, the dashboard, the vault, everything — resolves to that one IP.
So a single lost lease took out the entire front door and all of SSO at once,
while every backend service sat there perfectly healthy behind a door nobody
could reach. And because my monitors probe by hostname, they all dutifully
reported the wrong services as down.
The fix was to give the machine a real static address and a MAC reservation. But
the lesson is architectural and it's the most important one in this whole post:
Centralising auth centralises the blast radius. The thing that makes SSO
valuable — one door, one policy — is exactly the thing that makes one DHCP
hiccup an estate-wide outage. A single front door is a single point of failure
wearing a security badge.
I still run it single-homed, because for a homelab the operational simplicity is
worth it. But I run it knowing that's the trade, with the failure mode written
down, monitoring that can tell "the door is down" from "the house is down," and a
pinned address so DHCP can't move it under me again.
The security-first lessons, distilled
If you take the diagram from a hundred other homelab posts and add what those
posts leave out, you get this:
- "Mostly hardened" is a trap. A half-configured security tool is indistinguishable from a finished one until it fails. Audit the defaults you think you changed — especially the ones the example file is begging you to.
- Secure the chain, not the service. SSO is a pipeline: DNS → edge → proxy → Authelia → app → token verification. Each joint has its own failure mode, and the scariest ones don't report themselves as auth errors.
- Plan the way back in first. Before you disable local login, before you funnel everything through one provider — build the break-glass path and test it. Your security should never be able to lock you out.
- Verify secrets by fingerprint, never by printing them. The moment you echo a key to check it, it's in a scrollback, a log, a terminal-recording buffer.
- Know your single point of failure and monitor it honestly. If everything resolves to one IP on one box, a probe-by-hostname monitor will lie to you about what's actually broken. Make it able to tell the door from the house.
- Defense in depth is only defense if the layers agree. Two security systems in a row will eventually disagree about a request, and that seam is a bug waiting to look like an outage.
The architecture — one identity provider, 2FA everywhere, layered with edge
blocking — is genuinely the right shape, and I'd build it again tomorrow. But
"security first" turned out not to mean the clever OIDC config. It meant the
boring discipline around it: rotating a default I'd overlooked, keeping a way
back in, knowing which single machine could take the whole thing down, and being
willing to admit — in public, on the internet — that for a while my very secure
front door was locked with the key taped to it.
I wrote the longer version up as a field report — the full rotation procedure,
the break-glass pattern, the defense-in-depth seam, and a security-first
checklist you can run against your own single sign-on before you funnel
everything through it. It's
*pay-what-you-want, including free*.
Take the checklist, pay nothing if you like.
Homelab notes from someone who runs rather a lot behind one login and has
broken most of it at least once. Hostnames and IPs here are illustrative;
nothing about your setup should be guessable from a blog post, including mine.
🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.
Top comments (0)