My smart home runs on Home Assistant: lights,
sensors, a voice assistant, solar monitoring, a 3D printer, a small forest of
bulbs across a couple of VLANs. It's the classic homelab problem — the moment it
works, it becomes load-bearing for the household, and the maintenance never
stops. Devices drop off WiFi, an integration breaks after an update, a bulb goes
"unavailable" for reasons known only to itself.
So I wired an AI agent into the loop. Not "Alexa but nerdier" — an agent that can
actually look at the state of the system, run diagnostics, and do a bounded set
of fixes. This post is the honest breakdown of what I handed it, what I
deliberately didn't, and the rule that decides which is which — because "let an
LLM run your house" is a great way to end up with the heating on at 3am unless you
draw that line carefully.
The setup, briefly
Home Assistant runs on its own low-power box. The AI layer is a separate
self-hosted agent that talks to it through Home Assistant's own API and a couple
of purpose-built tools — it is not an integration living inside Home Assistant,
and that separation is deliberate. If the agent falls over, the house keeps
working; Home Assistant doesn't depend on it. The agent is a helper bolted
alongside, never a component in the critical path.
Under the hood the agent mostly runs against a local language model on a
separate machine, so the routine work — "summarise what's unavailable," "which
automations fired overnight" — costs nothing and sends no household telemetry to a
vendor. Cloud models only get involved for the occasional hard reasoning task, and
even then never with anything identifying.
What I handed the agent
The jobs I was happy to delegate all share a shape: reading state, and taking
actions that are easy to reverse.
-
"What's broken right now?" The single most useful thing. Instead of me
scanning a dashboard of 200 entities, the agent reports the exceptions: which
devices are
unavailable, which battery sensors are low, which integration reloaded itself overnight. It turns "stare at a wall of dots" into a two-line morning summary. - Triage of the flaky-device problem. Bulbs and plugs drop off constantly, and the causes are boring but varied — a device that's associated but never got a DHCP lease, one that's simply powered off at the wall, one stuck in a reconnect loop. The agent walks that ladder (is it on the network at all? does it have an IP? is it answering?) and tells me which kind of dead a thing is, which is 90% of the fix.
- Building tedious config from a registry. Dashboards, groups, repetitive automations — the agent generates them from a source-of-truth list I maintain, so the boring YAML is written for me and stays consistent. I review the diff; it does the typing.
- Explaining the smart home to me. "Why did the landing light come on?" is a genuinely hard question across a dozen interacting automations, and having something that can trace the trigger chain and explain it in a sentence is worth a surprising amount.
Notice what all of these are: diagnosis, summarising, and generating drafts I
approve. The agent is doing the reading and the typing. It is not, on its own,
deciding to do things to my house.
What I kept for myself (and why)
The line I will not move: the agent does not autonomously control the physical
environment. It doesn't set the thermostat on its own judgement, doesn't unlock
anything, doesn't decide the lights should be off because it inferred I'm asleep.
Anything with a real-world consequence that's annoying-or-worse to get wrong stays
either fully manual or driven by deterministic Home Assistant automations — not
by a model's in-the-moment call.
There are two reasons, and they're the same two reasons that run through
everything I self-host.
1. A language model is non-deterministic, and houses are stateful. An
automation that says "if motion and after sunset, light on" does the same thing
every time, and I can read it and know exactly what it'll do. An agent asked to
"manage the lighting sensibly" will do something reasonable-ish that I can't
predict and can't reproduce. For a blog draft, non-determinism is fine — you edit
it. For the heating while you're on holiday, it is not. Use the clever,
non-deterministic layer to inform and to draft; never to make a physical decision
you can't reproduce.
2. Fail-safe direction matters more than cleverness. When a deterministic
automation loses its inputs, I can make it fail in a known-safe direction (do
nothing, hold last state). When an agent loses context or misreads a sensor, the
direction it fails in is whatever the model felt like — and "the model felt like
turning everything off" is not a failure mode I want in the thing that controls my
front door.
So the division is clean: the agent is the eyes and the analyst; deterministic
automations are the hands. The clever part observes and advises. The boring,
predictable part actually touches the world.
The one that bit me: an outage that looked like a signal
The failure worth sharing, because it's the whole lesson in miniature.
I had the agent summarising smart-home health, and part of that summary leaned on
a data feed. When that feed had a brief outage, a null value fell through and
the summary confidently reported a specific state — as if the missing data were
real data. An outage rendered as a reading. It's the exact same bug I've hit in
completely different systems: when data goes missing, a naive pipeline makes it
look like a value instead of an absence, and everything downstream trusts it.
The fix wasn't cleverer AI. It was making the boring layer honest: when the feed is
unavailable, the state is unknown, full stop, and the agent is handed
unknown — never left to paper over a gap with a plausible guess. Which is,
again, the rule: the deterministic layer must report the ugly truth (unknown,
unavailable, stale), so the model has real facts to summarise instead of a
guess to launder into confidence.
Would I recommend it?
Yes — with the line drawn where I drew it.
An AI agent as the diagnostic and drafting layer over Home Assistant is
genuinely great. It collapses "scan everything" into "here are the three things
that need you," it triages the endless flaky-device churn, and it writes the
tedious config. That's real time back, every week, with essentially no downside —
because if it's wrong, the cost is a summary you ignore, not a house that
misbehaves.
An AI agent as the autonomous controller of the physical environment is a
solution looking for a problem. Deterministic automations already do the
controlling, they do it predictably, and they fail safe. Keep the model on the
side of the line where being wrong is cheap.
The smart home got genuinely easier to run the day I stopped asking "what should
the AI be allowed to do?" and started asking "what should it be allowed to
see and say?" — and let the boring, reproducible automations keep doing the
touching.
Homelab notes from someone whose house is run by predictable automations and
watched over by a chatty agent — and who is very clear about which of those two is
allowed near the thermostat.
🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.
Top comments (0)