The whole industry just figured out that AI agents lie about their own work. The part nobody's saying out loud: in a homelab, there's nothing standing between the lie and your infrastructure.
Jason Lemkin spent nine days building an app with an AI agent. He told it to freeze the code... no more changes, hands off. The agent changed it anyway, deleted his production database, then generated about four thousand fake records to fill the hole it had just dug, and when he asked what happened, it told him a story. That one made the rounds. It deserved to.
It's not alone. A report this spring found that 43% of AI-generated code still needs manual debugging in production... after it passed QA, after it passed staging. Another pegged AI-authored code at roughly 1.7 times the defect rate of the human kind. The phrase going around in February was "unreliable narrator." When the model says it's done, when it says it fixed the thing... you check. That need isn't going away, and anyone telling you otherwise is selling something.
Good. The industry's finally admitting the robot lies.
Here's what's been bugging me about every single one of those stories, though. They all happened somewhere with a net.
Lemkin had a production database to lose... which means he had environments, which means somewhere upstairs there was a staging tier and a deploy pipeline and, eventually, a human who noticed. The 43% number comes from shops with QA gates and the thing we used to call a code review: a pull request, sitting there, waiting for somebody to read it before it merges. The entire enterprise conversation about AI reliability is a conversation happening on top of guardrails. They're discovering the agent lies, and the guardrails are catching it. Late, expensive, three-redeploys-deep... but catching it.
You don't have any of that.
You wired an agent into your homelab because that's the whole point. It writes the compose file. It edits the .env. It touches the proxy config. It's supposed to harden the box, run the backup, and tell you it's done. There is no staging tier in your garage. There is no QA. There is no pull request and there is nobody to read it. It's you, a green dashboard, and a robot that just told you everything's fine.
Nobody's reviewing your robot's PRs. So who's calling it a liar?
Your monitoring isn't. Go read the homelab guides... the standard advice for "how do I know my AI services are up" is to point Uptime Kuma at them and check whether they respond. And there it is, the trap, baked right into the recommended best practice. Responding is not working. A thing can answer you and be stone dead behind the door... and your monitor will paint that green and call it a night.
I've got the receipts. Four of them, actually... I wrote the whole set up last time, over in Everything's Green Cap… Here's the one that still bothers me.
I hardened a Docker host's firewall... ufw, the tool everyone points you at, plus ufw-docker, because plain ufw can't even see Docker's published ports (Docker writes its own iptables underneath your rules... that's a different article). Ran it. Came back clean. Firewall: green. Except ufw-docker ships with a default that allows all of RFC1918... your entire private range, 10-dot, 192.168, the lot. On a flat LAN, that is not a firewall. That's a screen door reporting itself as a vault. Green status, open house, and not one thing told me. The rule was "active." The rule was a lie.
A screen door reporting itself as a vault.
It wasn't a one-off, either. A container reporting "up" while the actual service inside it crash-looped on a loop I couldn't see from the outside. A service that answered ping all day long... ICMP, healthy, fine... while the packet queue it was supposed to be reading sat dead and every real connection timed out. That's the exact failure your uptime-ping logs as "all good." And yeah... an agent that reported work it never actually did. The homelab-scale version of Lemkin's four thousand fake records. Smaller blast radius. Same lie.
Every one of those passed as green. Every one would have kept passing if I'd trusted the dashboard.
So here's the discipline, and it isn't a framework and it isn't a product you can buy. It's a posture.
Stop asking "is it up." Ask "is it doing the thing." Then go prove it from the side where it breaks.
Don't read the firewall rule... try the connection that's supposed to be blocked, from the place it's supposed to be blocked from, and watch it fail. Don't trust the backup job that exited zero... restore it, onto something real, and watch it come back. Don't believe the agent wrote the config it claims it wrote... diff what's live against what it says, byte for byte, and assume the bytes are the only thing in the room telling the truth. The health endpoint is the robot's narration. The behavior is the fact. When the two disagree, the behavior wins. Every time.
Status is the narration. Behavior is the fact.
I run a self-hosted SOC out of a 40ft fifth wheel with a stack full of AI doing most of the work. I'm not the guy telling you AI is bad... AI does about seventy percent of what happens on my rig, and I wouldn't have it any other way. I'm the guy telling you it lies. Constantly. Cheerfully. In green. And that the only thing standing between that lie and my infrastructure is me, looking at the thing, refusing to take its word for it.
The enterprise answer to all this is already shipping: bolt on an AI SRE... an agent that reasons over your observability and reassures you it's handling things. More green lights, narrated by a second robot, watching the first. I'm sure it'll be fine.
The homelab answer is older and cheaper and it doesn't scale, and that's fine, because you're not trying to scale. You're trying to be right. You look at the thing. From the failure side. Out loud, if you have to.
DIY or die. That includes not trusting the robot you built.
Top comments (0)