DEV Community: Määäx

My local AI ran on the CPU for a week and I didn't notice

Määäx — Fri, 07 Aug 2026 16:41:51 +0000

My first Ollama setup felt fast. One evening, one model, one question, a clean answer streaming back at a speed that made cloud AI feel unnecessary.

A few days later, with a bigger model and a longer question, the cursor crawled. No error. No warning. Just slow, in a way that made me suspect the model, the network, my own question — everything except the actual cause, which turned out to be almost embarrassingly mundane.

Here's what was actually going on, and the two things that caused it.

The setup

Ollama runs as its own service on Proxmox, with the GPU passed through instead of running on the CPU alone. The whole point of local AI, for me, is that no request ever leaves the house — but without usable speed, that's a nice principle nobody actually uses day to day.

In front of it sits a web UI for chatting from the browser; behind it, the Ollama API for whatever I automate later. Access works exactly like the rest of the homelab: the identity provider in front, reachable only through the tunnel, locked down the same way as everything else. No port opened just because it's "only my own chat."

The real work wasn't standing the service up. It was sizing it correctly — which model actually fits the GPU I have, what context length is realistic, and making sure the GPU Proxmox reports as "available" is the one Ollama actually gets.

Mistake 1: a model that almost fit — and Ollama never said a word about it

VRAM is a hard limit, not a soft warning. A model that fits entirely on the GPU runs there and runs fast. A model that almost fits gets partially offloaded to the CPU by Ollama, silently, with no error and nothing in the response that flags it happened.

That's what makes it dangerous. It doesn't feel like a failure — it feels like "a bit sluggish today," and a bit sluggish points you everywhere except the graphics card sitting there fully idle.

It got worse when I raised num_ctx — the maximum context length, how much text the model can hold in its head at once — because I wanted longer conversations. A bigger context needs memory on top of the model itself, and that extra chunk was exactly what pushed the whole thing past the GPU's limit, even though the model alone would have fit fine.

The lesson: model size and context length share the same VRAM budget, not separate ones. After changing either one, check actual GPU utilization — not just whether the answer "feels" fine.

Mistake 2: another VM already had the GPU, and nothing told me

GPU passthrough on Proxmox isn't multi-tenant. A passed-through card belongs to exactly one VM or container, never several at once.

I'd briefly set up a second VM using the same card for something unrelated, then just shut that VM down afterward instead of removing its passthrough config. On the next host reboot, that VM grabbed the card again automatically, before the Ollama container even started — and Ollama fell back to plain CPU inference without complaint, because from its point of view there simply was no GPU.

No crash, no warning. Just a service running roughly ten times slower than it should for the better part of a week, until I noticed while cleaning up the VM list.

The lesson: GPU pinning is exclusive and renegotiated on every boot. After a host restart, it's worth a quick check of which VM actually holds the card before blaming the AI service for being slow.

The setup, condensed

Same result, without the quiet CPU week:

Know the VRAM ceiling before pulling a model, not after. How much graphics memory is actually available, and how much does the model need at the quantization you picked?
Set num_ctx deliberately, not generously. Every increase eats memory the model itself needed.
Check actual GPU utilization after any change — not the response speed you feel, the number the hardware reports.
Assign GPU passthrough to exactly one VM, and fully remove test configurations on any other VM — don't just power it off.
Re-check which VM holds the card after every host reboot, before assuming the AI service itself is the problem.
Keep access locked down the same way as everything else — identity provider in front, reachable only through the tunnel, no exception just because it's "only the chat."

What changed

The gap between a local model that feels cloud-fast and one that crawls rarely comes down to the model itself — usually it's VRAM running out quieter than expected, or a card another VM already claimed. The reason for doing this at all hasn't changed: every request stays on my own network, nothing leaves the house, and that's worth the occasional re-tuning.

I write about self-hosting in more depth — one homelab component per issue, always including what broke — in my German-language newsletter Souveränes Homelab: souveraenes-homelab.beehiiv.com. The English write-ups land here.

Määäx

Zero open ports — my homelab is reachable from anywhere and exposes nothing

Määäx — Fri, 07 Aug 2026 13:14:41 +0000

My router forwards exactly one port to the internet. UDP, for WireGuard. That's it.

No port for the file storage. No port for the router in front of my services. No port for anything. A port scan from outside finds one thing it could try itself against, and that one thing only talks to devices holding the right key.

Getting there took two evenings of debugging, a phone battery that died twice as fast as it should have, and a tunnel that cheerfully displayed "connected" while passing zero packets. Here's all of it.

The moment the forwarding list got too long

After sorting out single sign-on (that's its own story — the short version: I locked myself out of my entire homelab), one login covered everything. But the port forwarding list on my router still read like an invitation.

File storage forwarded on 443. A second service on a custom port, because the first one had taken the obvious one. An SSH port I'd opened "just quickly" for an emergency and then forgotten to close.

Every open forward is a bet. I'm betting the service behind it is always patched. I'm betting there's no zero-day. I'm betting I never fat-finger a firewall rule.

Three bets I didn't want to keep taking. So: forwards out entirely, and exactly one encrypted way in instead.

What I built

The idea is simple, and that's the strength of it.

The router has one open UDP port for WireGuard and nothing else. Anyone who wants to reach my homelab first builds a WireGuard tunnel to the router — and is then, as far as the network is concerned, sitting in the same network as a device plugged in at home. The routing layer in front of my services (with the identity provider behind it) is reachable at its internal address exactly as before, but only through the tunnel, never from the open internet.

Every device — laptop, phone, my partner's tablet — gets its own key pair and its own peer entry in the router's WireGuard config. No shared credentials. If a device is lost or sold, I delete that one entry and nothing else is affected.

Same thinking as with SSO: not one shared secret for everyone, but an individual key per access path that can be revoked on its own.

Mistake 1: I accidentally built a full tunnel and broke my phone's internet

Setting up the first client config on my phone, I set AllowedIPs to 0.0.0.0/0, because that's what the first guide I found said to do.

What that actually means: not just the traffic to my homelab, but every last bit of the phone's internet traffic now went through the tunnel to my router at home — and only then back out to the internet from there.

On mobile data, the result was noticeably slower browsing, because every request took a detour through my apartment first, and a battery that drained about twice as fast, because the tunnel stayed up permanently for things that had nothing to do with my homelab.

It took me two days to notice that "my phone's internet has been sluggish lately" and "I set up WireGuard yesterday" were the same event.

The lesson: AllowedIPs is the knob between split tunnel (only traffic destined for the homelab network goes through) and full tunnel (literally everything). For day-to-day access to your own services you almost always want split tunnel — put your own subnet in there, not 0.0.0.0/0.

Full tunnel is a real and legitimate configuration. It's just a deliberate choice, not a default you copy out of a guide.

Mistake 2: "connected", and not a single packet got through

Out on mobile data — behind the carrier-grade NAT most mobile networks use — WireGuard completed the handshake cleanly. Then, after a few minutes of sitting idle, the connection outward was effectively dead. The app still said "active". No ping, no page load, nothing.

No error. No message. Just silence.

I blamed the router first. Then DNS. Then, eventually, I found the detail that the first guides I'd read barely mentioned:

Mobile carrier NAT forgets open connections far faster than a home router does, and WireGuard, by default, sends nothing to keep the mapping warm when there's no data to move. No traffic, no packets, and the NAT entry quietly expires. The client has no idea — from its point of view the tunnel is still established.

The lesson: set PersistentKeepalive to a low value (25 seconds is the common choice) for any client that sits behind NAT — which in practice is every mobile device. It sends a tiny packet during idle periods so the NAT mapping stays open and the tunnel is genuinely reachable, not just "connected according to the display".

That distinction — reachable versus shows as connected — is worth internalizing. It shows up in a lot more places than WireGuard.

The setup, condensed

Same result, without the two evenings:

Open exactly one port. UDP for WireGuard on the router. Everything else that was previously forwarded: closed.
One key pair per device. No shared setup, every device gets its own peer entry so individual access can be revoked individually.
Scope AllowedIPs deliberately to your homelab subnet (split tunnel), not 0.0.0.0/0 — unless routing all your traffic is what you actually want.
Set PersistentKeepalive = 25 on every device behind NAT — that's essentially every phone.
Hand over internal DNS, so hostnames on the homelab network resolve over the tunnel too, not just raw IPs.
Test from real mobile data, not from your own Wi-Fi. Mistake 2 only surfaces out there; at home it would never have shown up.
Keep the routing layer and the identity provider exactly as they are. The tunnel replaces the way in. It does not replace authentication behind it.

That last point matters. A tunnel is a network control, not an access control. Anything on the other side that was worth protecting before is still worth protecting.

What changed

The difference isn't "faster" or "more convenient." It's invisible, and that's the whole point.

From the outside, my homelab shows no service port at all anymore — just one encrypted tunnel endpoint. There's nothing there for an automated scan to work with, and getting through requires a key I handed out myself, to a device I can revoke individually.

The old setup wasn't reckless. It was just a pile of small bets I'd stopped keeping track of.

Määäx

I put SSO in front of eleven self-hosted services — then locked myself out of all of them

Määäx — Fri, 07 Aug 2026 13:10:42 +0000

Eleven self-hosted services. Eleven logins. And — being honest here — at least four of them had a password I'd have described as "good enough, I'm the only one who gets in anyway."

That sentence is the beginning of the end.

There was one evening that broke it for me. I wanted to move a single photo from Nextcloud into Paperless. One handgrip. It cost me three different passwords, plus the container dashboard on top, because something in the stack was acting up and I needed to see why. A homelab doesn't grow according to plan — it sprawls. And at some point user management stops being a detail and turns out to be the foundation you forgot to pour.

So I made a decision a lot of people find excessive: before finishing the rest of the homelab, I put a central identity layer in front of it. Single sign-on with Authentik. One login for everything.

Overkill for a home network? I didn't think so. But it did lock me out of my own house, and it did cost me two evenings of chasing a bug that turned out to be one missing thing. Both are worth writing down, because both are the kind of mistake the tutorials skip.

What I actually built

Authentik is a self-hosted identity provider. The mental picture that works for me: it's the office door with the one badge reader, and every room is behind it.

It speaks the usual languages — OpenID Connect and SAML for apps that support them, and forward auth through the router that sits in front of my services for everything that doesn't. In a homelab, "everything that doesn't" is most of it.

Concretely it runs as a Docker stack: Authentik server, worker, PostgreSQL, Redis. In front of that sits the edge router (Traefik-style routing — Nginx or Caddy do the same job) that takes every request for a protected service and asks Authentik first: is this person allowed in?

Not signed in → you land on the login page. Signed in → you get waved through, and the app is told over a header who just walked in.

The end state: one login for eleven services, 2FA in exactly one place, and when I want to give someone — partner, family — access to exactly one service, I do it through a group instead of handing out another shared password.

Mistake 1: I locked myself out. Completely.

When I set up the forward-auth rule on the router, I was thorough. Too thorough. I put everything behind Authentik — including Authentik's own login page.

Read that again slowly, because that's exactly how long it took me to see it: in order to sign in, I had to be signed in.

A perfect infinite loop. I sat in front of a redirect pointing at itself and could not reach a single one of the eleven services. Not the dashboard, not the file storage, not the thing I'd been trying to fix in the first place.

What saved me was that I could still SSH into the host and pull the routing rule for the Authentik host back out.

The lesson has two halves, and the second one is the important one:

The identity provider and its own login route must never sit behind the auth it provides. Exclude it explicitly.
You always need a way past the front door. An SSH path to the host that does not depend on the exact system you're currently rebuilding. Check that it's open before you touch the auth layer, not after.

That second point generalizes far past SSO. Any time you're putting a new gate in front of everything, the first question is: what's my way in if the gate is broken?

Mistake 2: SSO "worked", and every app thought I was a guest

Once the login finally stood up, some services worked immediately. One let me through — and then showed the user as an unnamed guest. Everywhere. Every page, no name, no groups, no permissions that matched who I actually was.

I spent two evenings on this. I suspected the app's config. I suspected Authentik's mappings. I re-read the docs twice.

The cause was banal and, I've since learned, extremely common: the router validated Authentik's answer, but never passed the headers with username and groups on to the app.

So the application received a clean "authenticated: yes" — and no name. With no name, it did the only sensible thing it could and fell back to its guest user.

The lesson: forward auth is two stages, and they fail independently.

Stage one — may this request in? That's the auth check. When this breaks, you notice instantly, because nothing loads.
Stage two — who is it? Those are the headers passed from the auth endpoint to the actual application. When this breaks, everything looks like it works.

If SSO is "sort of half working" — you get in, but the app doesn't know you — it's almost always stage two. Go look at your header pass-through before you look at anything else.

How I'd set it up today

Same result, without the lockout and without the two evenings:

Start Authentik in isolation first. Server, worker, database, cache as their own stack. Test the login while nothing sits in front of it.
Secure a second way in. SSH to the host, independent of SSO. That's your fire exit. Verify it's open before every auth change.
Connect exactly one unimportant service. Not all eleven. One test service where you get the forward-auth pattern right in a place where breaking it costs you nothing.
Verify both stages separately. Does a signed-out visitor land on the login page (stage: may they in?), and does the service then show the correct username (stage: who? ↔ the headers).
Exclude the identity provider itself. Explicitly carve the Authentik login route out of the auth rule. Otherwise: loop.
Only then roll out — service by service, signing out and back in after each one.
2FA and groups last. Second factor at the one door, access through groups instead of shared passwords.

What actually changed

The real gain wasn't convenience, which surprised me.

It was that for the first time I had one place where I can see and control who gets into my homelab — and that I can revoke someone's access with one click, without having to remember eleven separate places where I once gave it to them.

Building SSO before the rest feels like pouring a foundation when you'd rather already be putting up walls. That's exactly why it holds later.

Määäx