Michael Brewer

Posted on Sep 11 Originally published at michaelbrewer.me

Case study: a household of agents

#ai #agents #automation #productivity

Six months of records from a multi-agent stack that runs part of my life, and what the record, not my memory, says worked.

Summary

I run a set of AI agents on a small dedicated machine in my house. Each one owns a domain: a general assistant, the homelab, and a handful of hobbies and volunteer commitments. They talk to me over chat, run scheduled jobs, read email, and ship deliverables I can open myself. I installed the platform in March 2026, after seven months of chasing the agentic coding wave: every hyped-up harness that came along, plus an orchestrator of my own. The stack running today was the fourth build in its first week, and this is what its own records say about the six months since.

The short version is uncomfortable. The value never came from how smart the models were. It came from how fast I could turn a lesson into machinery: a script, a state file, a cron job, a hook, a checksum. Every durable win on this system is deterministic plumbing with a language model sitting at one narrow seam. Every recurring failure came from asking a model to do something it cannot be trusted to do on its own word: hold state, remember what it already knew, verify its own work, stay in its lane, or stop when told.

After six months, the most important thing I have built is not an agent. It is the enforcement and verification layer around the agents, and the habit of never believing "done" until I can see the artifact.

The problem

I built this because of a specific weakness that I asked the agents to cover. I am good at starting things and weak at maintenance, follow-through, and impulse decisions. A full-time job, a family, a two-acre property, a homelab, and a board seat produce a constant stream of small obligations, and the twenty minutes each one needs never lines up with the twenty minutes I have. The agents were supposed to own the follow-through.

The seven months before the install had already taught me one thing. It is entirely possible to spend ten times the hours building guardrails for a model as the hours it saves you on the project. I installed it anyway, because the need was real and I wanted to find out whether a platform built for the job would change that ratio.

The first three attempts, all in one week in March, failed the same two ways. They talked about work instead of doing it, and they did not finish. Each rebuild changed the isolation, the model, or the prompts. None of that fixed it. The fourth attempt is the one that stuck, and the difference was not a better model. It was that enforcement shipped on day one.

Constraints

Cheap models on purpose. Early research put frontier models at roughly $47 a week for this workload against about $6 for a budget cloud model. The whole point was always-on, so the budget model won, along with every weakness that came with it.
Every agent has a shell. That is what makes them useful, and it means the worst days were never "the agent said something wrong." They were "the agent did something irreversible."
One on-call engineer. A personal agent stack is a production system with a staff of one. The platform under it has bugs that look exactly like agent misbehavior, and I am the only one who will ever notice either.
The agents live on a firewalled subnet, on purpose. The agent machine sits on its own isolated network behind carrier-grade NAT. It cannot reach the main LAN or the public internet except through the paths I gave it, and that was the right call before the first agent ran. What the record shows is that the agents kept trying to configure around it, planning for public traffic that could never arrive and breaking things in the attempt, until that boundary became the first section of their memory instead of an obstacle to route around. The friction was theirs to absorb, not mine to remove.

Options I rejected

An orchestrator that routes with a language model. The architecture I settled on for the fourth build, and the one the community recommended, had a manager agent read every message and delegate to specialists. It lasted two weeks. It editorialized instead of dispatching, its delegation calls failed silently when the model left out the target, and the runtime it delegated through had a lifecycle bug where results never came back. Routing is now a lookup table: one chat topic per agent. No model sits in the routing path.

Rules in capital letters. Every workspace is full of bold rules written the day something went wrong. One rule was violated more than 30 times after it was written. Another set of four "fireable offense" rules was broken, all four, at least once afterward. Around the second week I stopped believing that a rule in a prompt would be followed because it was emphatic.

Letting the research agent's knowledge base live inside the agent. A research agent ran every 30 minutes and built a body of research-backed material that now covers just about any question in its domain: hundreds of runs at a 96 percent success rate and a vector memory assembled with zero human involvement, on tokens cheap enough to make that possible. The agent was doing exactly what it was supposed to, and the value was real. I turned it off because the knowledge base grew large enough to put memory pressure on the whole system and to bloat the agent's own context on every turn, which broke the narrow-domain ethos the rest of the household runs on. The right shape is a spin-off knowledge base with its own dedicated multimodal RAG setup that the agent sends requests to instead of carrying the material itself. Whether that gets built, time will tell.

Long fallback chains. A seven-model fallback chain hid a three-month outage. The local model provider had never served a single successful request, because a base URL was missing a path segment, and the chain quietly skipped past it every time. A failover chain is a way to hide outages from yourself.

The model as formatter. A quiz script produced a question as JSON and the agent formatted it, badly, twice. The fix was not a sterner prompt. The command was deleted and the script now sends straight to chat with hardcoded formatting. The agent has zero formatting vector.

What shipped

Verification that lives outside the agent. An agent's report about its own work is a second artifact that may or may not match the first. A printed checklist once showed 58 of 95 items, was called complete, and was defended three times before anyone looked at the PDF. Now no rendered deliverable leaves without every page converted to an image and the rows counted, and the delivery message carries the proof. A presubmit script blocks files that violate a standing format rule. A media archiving pipeline ends in a verdict script that will not pass an item unless an independent database confirms the data or a recorded human approval overrides it, with three separate fixity records per item.

Hooks instead of prompts. The comment at the top of the first enforcement plugin states the whole problem: the platform can restrict what agents can't do, but cannot force what they must do. Fourteen enforcement plugins now cover that gap. One detects action requests and blocks chat-only replies. Several force a memory search before any action and a memory write before completion. One blocks replies that tell me to clear my cache or try again. One blocks mutations on external surfaces until declared sources have been read. One blocks bulk rewrites and non-English script in memory. One blocks the agents from upgrading the platform under themselves. Email goes through a relay that rejects any non-allowlisted recipient before a connection opens, with the allowlist hardcoded so an agent that controls the environment cannot widen it.

Memory with a schema and a size cap. Memory was the most dangerous component in the system. An overnight planning job produced over 200 files and 43,000 lines in four days, mostly regenerated copies of the same plan, with confident wrong claims buried inside that memory search kept resurfacing. The fix was deleting it and rebuilding by hand. Memory is now one file per topic, a check script hard-fails any index over 16 KB or any duplicate heading, reflections go to dated files, and the best-behaved agent's entire memory sits near 5 KB with each task written as a command plus a "done when."

The model at the narrowest seam. The things I would miss tomorrow are almost all shell scripts with a language model doing one small job, or no job at all. A daily reminder is a cron job that sends buttons and writes a state file. Follow-ups fire every 15 minutes until acknowledged. When I tap a button, a plugin updates the state and replies. No agent turn runs and no tokens are spent per tap. A weekly config sync diffs a proxy against the config, asks the model to review the diff, snapshots, applies, and notifies. On its first run the review rejected a halved token limit. The model earns its place as a reviewer of a diff, not the author of the change.

Blast radius capped mechanically. After a "clean up the non-English text" request became a character filter that destroyed 1,089 memory files in one pass, and the same misreading produced 106 global edit commands three days later, the rule became structure: cleanup means rewrite, stop and ask above five files, and a daily sweep that reports stray text and never edits anything. Both incidents were survivable because a backup existed.

The same discipline for the builder. The coding assistant I used to build the stack failed in exactly the ways the agents did. It answered before reading, claimed verification that never happened, took conversational agreement as permission to edit 35 live cron jobs, and bypassed its own guardrail within minutes of writing it. One measured comparison: the rushed fix for a gateway restart problem took 106 tool calls and destroyed the install. The careful fix, three lines in a shell profile, took 60. It now runs under nine hooks of its own, including a snapshot before any config change and a rule that nothing is done until the change is tested through the gateway.

Outcome

Measure	Value
Scheduled jobs defined and enabled	77 and 69
Scheduled runs recorded, March 24 to September 10	14,563
Runs that did not finish clean	1,974, or 13.6 percent
Model turns in one measured week	6,308, about 900 a day
Input tokens in that week	33.7 million
Turns in that week that hit the prompt cache	35, or 0.55 percent
Enforcement plugins, March to September	5 grew to 24 enabled entries, 14 behavioral
Items in a verified media archive, closed September 7	95

What I would miss tomorrow: the reminders, the verified archive, travel support that kept my records straight through a canceled flight, working from the changes I told it about and the itinerary emails I forwarded, and rebuilt the documents, an eight-week volunteer curriculum published on schedule, a daily exam-prep quiz, email triage across five inboxes, and a 7 AM homelab check that reads the gateway log and names real problems.

What never shipped: a finance agent that was created and never bootstrapped, and an A/B switch built in July to test whether lean prompting on a stronger model could replace the enforcement stack. It was validated and never flipped, and not from neglect: the enforcement stack and the proxy refinements started paying dividends in quality and took the pressure off the simplify-and-hope option. That option is what every frontier model quietly wants, to define for itself what counts as done, so of course a model suggested it.

The cache number is the one that stings. Routing through the quality-gate proxy rewrote payloads and defeated the vendor's automatic caching, so one measurement showed 276 turns and zero cache hits, roughly 600,000 input tokens paid twice. The enforcement injections themselves cost context on every turn. Coercion is not free.

What I would do differently

I would not spend my own hours writing documentation for the agents. Every rule I put in prose was a bet that the model would read it, weigh it, and comply, and the record above shows how that bet paid. I would go straight to more coercion and more constraints: hooks that block, scripts that check the artifact, allowlists hardcoded where an agent cannot reach them, and a size cap on memory from the first day. A constraint costs an afternoon once. A hundred failed attempts cost an afternoon each, plus the cleanup. The constrained version is the cheaper product, and it is the only version that ever finished anything.

A personal agent system spends the operator's attention to save the operator's attention. It is only a win when the second number is bigger. For the first few months, it was not. What moved the balance was never a better model. It was moving each recurring correction out of my head and into a mechanism, so I stopped making the same correction twice. The ten-to-one ratio from before the install did not go away. It got paid down, one mechanism at a time, until the machinery started saving more hours than it cost.

DEV Community