DEV Community: Michael Brewer

My First User Said "Yeah, Absolutely"

Michael Brewer — Sun, 18 Oct 2026 13:14:06 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

replydesk drafts replies to a small business's reviews in the owner's own voice.

Reviews sit in public under a business's name, and the next customer reads the replies. Writing them is the hard part. The first owner who tried replydesk put it plainly: "I suck at putting all that together."

Here's how it works. You sign up at replydeskhq.tech, write a few lines about how you talk (the tone guide), and paste in your reviews. replydesk sorts each one into thank-you, fix-it, or needs-a-human. Legal threats, safety or health claims, refund disputes, and anything personal go to needs-a-human. Those never get a drafted reply, only a short note saying why. Everything else gets a draft from GLM-5.3-Flash, an open-weight model released under the MIT license.

Before you see a draft, a checker reads it against your tone guide, the review, and your business profile. If the draft makes a promise, quotes a price, names a person, or states a policy that isn't in one of those, it goes back to the model with the problem quoted. If the second try fails too, you get a template reply with a label on it instead of a made-up fact. Either way, you edit the reply, copy it, and post it yourself. replydesk never posts anything for you.

It's for solo owners and small shops, where answering reviews is real work and nobody's actual job. The founding plan is $9 a month, and my first user signed up for it.

Demo

The product is live at replydeskhq.tech. Sign up, start the founding trial (no card), paste in a few reviews, and draft replies.

If you'd rather not make an account, there's a recorded run at replydeskhq.tech/demo. It's the same pipeline run against Ridgeline Plumbing, a made-up company with reviews written for the project. A fresh clone runs the same demo with no key: py app.py --demo.

If live drafting is ever down, the app falls back to template replies, each one labeled [Template reply - edit before posting].

Code

dev-brewery / replydesk

Review replies in your own voice - open-weight GLM-5.3-Flash, triage + fact-grounded drafts. Live: replydeskhq.tech

replydesk

A review-reply desk for small businesses: paste the raw copy from your Google Business Profile or Yelp page, and replydesk triages every review (praise mixed, needs-a-human) and drafts a reply in your own voice — grounded in your tone guide, with a verifier that rejects any draft inventing facts your paste never contained. The offline core runs entirely from committed fixtures, and a recorded demo renders the full flow without a network. Hosted demo and help replydeskhq.tech · recorded demo · help.

The open-source core is model-agnostic: it ships pointed at GLM-5.3-Flash (MIT weights) but speaks to any OpenAI-compatible endpoint — set LLM_BASE_URL and LLM_API_KEY in a gitignored .env and it runs.

Quickstart

Python 3.12, standard library only — no dependencies to install.

# offline demo, no key needed
py app.py --demo
# online mode: copy .env.example to .env and fill endpoint + key
py app.py --online

#

…

View on GitHub

Python 3.12 and the standard library, with no dependencies, plus plain JavaScript for the site. MIT licensed.

How I Built It

There are two sets of models in this project: the one inside the app, and the ones that built it.

The model in the app. GLM-5.3-Flash writes the drafts, through any OpenAI-compatible endpoint. On the live site, the browser talks to a small API on an Oracle Cloud ARM VM, and that API calls GLM-5.3-Flash on z.ai. The model's job is narrow on purpose, and two pieces of plain code keep it that way.

The first is triage. A rule table decides what needs a human. The model gets one small say: when no rule matches and the star rating is all that's left to go on, it can move a review between thank-you and fix-it. In the recorded demo, a four-star review says the crew had to come back a second time to finish a water heater swap. The stars said thank-you, and the model moved it to fix-it, which is right. Escalation is different. The review that mentions a lawyer goes to a human because the word matched a rule, and nothing the model writes can talk the app out of that.

The second is the checker. In the run that produced the demo, four drafts passed on the first try, one passed after the checker sent it back, and one failed twice. The one that failed twice was the water heater review, so the demo shows the labeled template in its place.

The models that built the app. replydesk was built by three roles in pi, an open-source coding agent. Each role has its own model and a written charter, and all three ran on z.ai:

An orchestrator (GLM-5.3, max thinking) turned the task list into one brief at a time, launched the builder, checked each result against the acceptance criteria before writing the next brief, and kept the journal.
An architect (GLM-5.3, high thinking) wrote the design and the task list.
A builder (GLM-5.3-Flash, low thinking) wrote the application code and its tests, one task per commit.

I made the product calls. The first version had you paste reviews in one at a time, and I told the orchestrator nobody would pay for that. Then I told it the app had to be ready to sign up a real user, hosted on the real domain. By the time my first user sat down, it had sign-up, the founding plan, a parser that reads a raw copy of a Google reviews page, and live drafting at replydeskhq.tech.

The test suite has 303 tests and runs offline with the network mocked. That number means less than it sounds like, and My Agent Session below is why.

Why Does Open Innovation Matter?

Reviews are full of other people's information: names, complaints, what went wrong in somebody's kitchen. A tool that answers them reads all of it.

Right now the live site sends it to z.ai's hosted GLM-5.3-Flash. That's a setting I can change. The weights are open and replydesk talks to any OpenAI-compatible endpoint, so moving the model is three lines in a config file: LLM_BASE_URL, LLM_MODEL, and LLM_API_KEY. A shop, or someone who sets this up for a handful of local shops, can run the app and the model on their own hardware, and the reviews never leave it. And if z.ai's pricing changes, the same weights can be served by someone else, so a $9 plan doesn't depend on one company's rate card. A closed API can't make either offer. Your customers' reviews go wherever its model lives, at whatever price it sets.

The build is open the same way. The team runs in pi, and every task went to the builder as a brief written to a file. If the builder did something strange, I could open the brief and read exactly what it had been told.

My Agent Session

The whole build is on the record in JOURNAL.md: the tasks the orchestrator handed out, the checks it ran, the calls I made, and the places the agents got it wrong.

A few of the bugs in that journal passed every test and broke the real thing.

The app that didn't start. For the better part of a day, py app.py did nothing. It had no __main__ guard, so starting the server was a silent no-op, and the suite stayed green the whole time. It turned up when the orchestrator ran the app for real.
The drafts that died in the browser. Every live draft in Chrome failed with "Illegal invocation," because the code passed fetch around detached from window. The tests hand the code their own fetch, so they never saw it.
The .env that wasn't there. Part of the suite only passed on a machine with a local .env. The final audit runs the tests on a fresh clone without one, and it caught that on its first run. The same blind spot came back on the move to the VM: the container has no .env on purpose, and the config loader crashed looking for the file before it ever checked the environment.

The builder's first try at committing that audit put real values from my privacy patterns into its tests. The pre-commit privacy guard blocked the commit, and the builder rewrote the tests with made-up values instead of going around the hook. That's what the hook is for.

Before I published, I had Claude Code review the repo against the challenge rules, read-only. It came back with twelve findings, and the three serious ones were all things the agents had written down as true that weren't. The draft of this post said every commit carried the builder's sign-off, and 24 of 49 didn't. It said the app ran on my own hardware, when the model calls went to z.ai. And the recorded demo labeled the water heater template as a verified model draft, which is the one thing the checker exists to prevent. All three are fixed, and the fixes are in the journal too.

My First User

A local business owner tried replydesk on his own reviews. We signed him up at replydeskhq.tech, started the founding trial, and worked through a real review from his Google Business Profile. I recorded the session, 22 minutes of it, and transcribed it on my own machine with faster-whisper, an open-source speech model, in about two minutes.

His first draft came back wordy. I asked what he thought, and he said it was "a fair amount of almost, like, saying it twice." So I had him go back to the voice section of his tone guide and add a few words: don't be too wordy, get to the point, and always sign off with "God bless." He hit draft again. The new draft came back quicker and signed off with "God bless."

"See, I like that. I suck at putting all that together. But now reading this..."

Putting the words together is the part he doesn't like, and a couple of lines in a text box were enough to get the drafts close to his voice.

Did he pay? Would he? Not yet, because there's no billing to pay through. When we got to price, I told him I was thinking $9 a month, enough to cover the AI behind it with a little margin. "Yeah, absolutely. And then we just have to set something up, you know, a payment plan." He's on the founding trial now: $9 a month, invoiced later, no card up front.
What did he ask for next? A way to know when a review comes in. "How am I going to get notified that I get a new review? That's going to be my catch." He was worried about remembering to come back, and a watcher that pings him about new reviews is first on the list.
Where did it rub? The paste. He copied his reviews from his Google Business dashboard, which formats them differently from the public Maps page (the stars come through as the literal word "star"), and the parser had only been built against the public page. He typed the review in as a 5 | review text line instead, and that worked. Once I had the exact text that failed, the builder taught the parser the dashboard format the same afternoon, and the help page covers it now.

His name and his business stay out of this post and out of the repo, and so does the recording.

Where It Stands

replydesk is five days old. Accounts live in your browser for now, so they don't follow you to another device. A server-side account system is already running behind api.replydeskhq.tech for when billing arrives, but the site doesn't use it yet. Until then, founding members get invoiced. New-review notifications are next.

The parts of replydesk I trust are the parts where something other than the model checks the model's work. That goes for the app and for the team that built it.

Prize Categories

Best Use of a .tech Domain. replydesk lives at replydeskhq.tech, a domain I claimed through Hacktoberfest. It serves the product itself: the landing page, sign-up, the founding trial, and drafting all happen there. The recorded demo is at /demo, the help page at /help, and the API at api.replydeskhq.tech.

clearnight: a stargazing planner that tells you which nights are worth going outside

Michael Brewer — Sat, 10 Oct 2026 15:56:22 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I built and how it gets people outdoors

clearnight is a stargazing planner that answers one question for the week ahead:
which nights are actually worth going outside for — and what will be up there
when you are?

It fuses three kinds of data into a ranked list of nights:

The sky — astronomical darkness windows (Sun below −18°), Moon phase, illumination, rise/set and altitude through each window, active meteor showers with radiant-up hours, and visible ISS passes (sunlit station, dark observer, max altitude, direction of travel). All geometry computed locally with skyfield and a cached JPL ephemeris.
The weather — keyless Open-Meteo hourly forecasts sliced to each night's dark window: cloud layers, humidity, visibility, dew point, wind.
A sensor-shaped local feed — the app reads whatever serves current site conditions as JSON over HTTP: a real instrument, a Home Assistant entity, or (as in this proof of concept) a mock file with preset values through the same adapter. It feeds a dew-risk estimate and a pressure trend, and a source-agreement view against the forecast. The adapter is the point: any JSON source plugs in with a config change and zero code.

Each night gets a deterministic 0–100 score with a per-factor breakdown and a
plain verdict — PRIME, WORTH IT, Marginal, or STAY IN BED — and the plan renders
as a red-on-black single page you can read on a phone in the dark without
ruining your night vision.

It gets people outdoors by removing every excuse: the exact window, the exact
things to look for, and honesty when the answer is "stay in bed." And it holds
itself accountable — a built-in field log records what actually happened, and
the next plan shows its own past predictions next to your outcomes.

Demo

The full offline plan — seven ranked nights, verdicts, the Draconids peak
marked, an honestly overcast weekend capped at STAY IN BED, and the labeled
brief:

A night's prediction rendered beside its logged outcome — the accountability
loop, demonstrated with a mock log entry:

Both regenerate from a fresh clone: python -m clearnight plan --offline.

The verifier in action. When the local model wrote a brief that corrupted
the dark-window times, the plan refused to show it — the retry loop fed the
violations back, and the surviving brief cites only computed values:

Go out at Black Mesa State Park (demo) on 2026-10-08 from 8:55 pm to 6:25 am.
Face E, S, or SE. Look for Draconids, ZHR 10, and Orionids, ZHR 20. New Moon,
2% illumination. Cloud 0%, humidity 44%, wind 11.8 km/h, visibility 43.0 km.
Score 100, PRIME.

Footer: brief: llm-verified (2 LLM call(s), 17 claims checked).

Accuracy. Computed dark windows match timeanddate.com for the demo site
within 1 minute; the Oct 9 near-new Moon (never above the horizon all night)
matches the independent reference. The cross-check ships as a test
(tests/test_accuracy.py), not a claim.

Code

dev-brewery / clearnight

clearnight

A stargazing planner that ranks the next week of nights for your site. For each night it computes astronomical dark windows, Moon phase and rise/set, ISS passes meteor showers, and the cloud forecast, then scores and ranks the nights with a clear verdict (GO / MARGINAL / NO-GO). It layers in local sky-sensor readings, a grounded LLM observing brief that is verified against the computed numbers, and a field log that records what actually happened so you can compare predictions to outcomes over time.

Install

Requires Python 3.12+.

git clone <your-fork-url> clearnight
cd clearnight
py -3.12 -m pip install -e .

Notes:

On Windows, the tz database comes from the pinned tzdata package (already in the dependencies; no extra step).
The astronomy engine is skyfield, pinned to 1.49.0 for reproducibility.

Quick start (offline demo, ≤ 5 commands)

git clone <your-fork-url> clearnight
cd clearnight
py -3.12 -m pip install

…

View on GitHub

github.com/dev-brewery/clearnight (MIT). Python 3.12, one pinned runtime
dependency (skyfield) plus a Windows tzdata marker, standard library for
everything else. No API keys for any data source, no accounts, no telemetry.
python -m clearnight plan --offline reproduces the demo plan from committed
fixtures with zero network — verified from a fresh clone. A 269-test suite runs
fully offline. Every commit is authored and signed off by the builder model,
and the complete delegation journal — every decision, brief, verification, and
correction — ships in the repo as JOURNAL.md.

How I built it with open-source AI

The entire codebase was written by an open-weight model working as a delegated
builder — every commit authored and signed off by the model, verified by a
separate model acting as orchestrator. I made the decisions and then stepped
out of the loop entirely.

The team is four roles wired into the coding agent (pi), each with a committed
charter:

Owner (me): decisions only — sensor, site, scoring weights, verifier strictness. Seven determinations, then hands off.
Orchestrator (GLM-5.3, max thinking): turns the task list into one-task briefs, launches builder sessions non-interactively, independently verifies every acceptance test before the next brief, and journals every outcome — including its own mistakes, which are on the record.
Architect (GLM-5.3, high thinking): designed the modules, the exact scoring formula, and the verification rules — and surfaced every owner-level decision as options before embedding anything.
Builder (GLM-5.3-flash, low thinking): wrote all code, one task per commit, tests with every module.

Open-source AI is also inside the app, not just around it: the observing
brief is written by an open-weight model — but every time, number, and event
name it writes is checked by a verifier against the computed data. A brief
that invents anything gets sent back with its errors listed; if it keeps
failing, the app falls back to a clearly labeled template. The LLM never
computes the score. The deterministic core produces the numbers; the model
only turns them into words, and it is not trusted.

Why open innovation matters for this project

Your location stays yours. Coordinates and endpoints live only in a gitignored local config; the committed demo uses a public dark-sky site (Black Mesa State Park, Oklahoma). This proof of concept even ran its sensor feed from a local mock file — the whole demo reproduces with zero network.
The model is swappable by design. The in-app brief writer speaks to any OpenAI-compatible endpoint — a local llama.cpp server with no key, or a hosted provider with one — configured in one line. Open weights mean the app still works when the cloud doesn't.
Keyless open data. Open-Meteo forecasts, CelesTrak orbital elements, JPL ephemerides, the IMO meteor calendar — no accounts, no keys, no telemetry, and every network call has a timeout and an offline fallback.
Verified generation, not trusted generation. The verifier pattern — generate, check every claim against computed truth, retry or fall back — is how small open models become reliable enough to ship, and it's all in the open.

The outdoor loop, demonstrated with mock data

Full honesty up front: this is a proof of concept, completed with mock data —
no field test was conducted before the submission deadline. What that means
concretely:

The sky, weather, and scoring are real computed data for a real public site — the planner genuinely says the Draconids peak night (Oct 9) carries 9.6 hours of astronomical darkness with a Moon that never rises into it, under a fixture-recorded 0% cloud cover, scoring it PRIME.
The sensor feed is a mock: an HA-shaped JSON file with preset values, read through the same adapter a real instrument would use.
The prediction-versus-outcome loop is demonstrated with a mock log entry (second screenshot above) — the feature works exactly as designed; the entry in it is labeled demo.

In a real deployment you carry the plan outside, and python -m clearnight log --date YYYY-MM-DD --went-out --clearness 4 --meteors 12 records what the sky
actually did, right beside what the planner predicted. The loop that keeps the
tool honest is built, tested, and demonstrated — the night out is left as an
exercise for the reader, ideally on a PRIME one.

Built in ~48 hours by a four-role open-weight AI team with a human owner who
only answered questions and clicked nothing else. The full record — every
decision, every brief, every correction, including the orchestrator's own
misreads — is in the repo's JOURNAL.md.

Case study: photoframe 3.0.0

Michael Brewer — Wed, 07 Oct 2026 15:42:52 +0000

Thirteen months from forking a Raspberry Pi project whose photo source disappeared to its first stable release, and why the release only counted once a real Pi had run it.

Summary

photoframe is Henric Andersson's Raspberry Pi project that shows random photos from Google Photos like a picture frame. In 2025 Google cut the Photos Library API down to photos an app uploaded itself, and the frame had nothing left to read. I self-host Immich, so I forked the project in September 2025 to teach it Immich.

Credit where it belongs: Henric Andersson (mrworf) started photoframe in 2018, shipped releases through v2.0.2, and kept supporting it for years after, with comments on 174 issues and pull requests and commits as recent as 2025, including the Python 3 branch this fork builds on. He kept the project alive long before AI tools gave me the time to resuscitate it.

On October 6, 2026, the fork shipped v3.0.0, its first stable release: Python 3, Immich as the main photo source, display handling for current Raspberry Pi OS, and a flashable SD card image. Against the original project's master that is 65 files, 4,686 lines added and 1,662 removed, across 43 merged pull requests.

The code was written with AI agents. The release discipline was not optional, and it came from one bad morning in April: nothing gets pushed, tagged, released, or sent upstream until it has run on Pi hardware. The worst bugs in this write-up were caught by that rule.

I covered the early part in The Photo Frame I Didn't Have Time to Fix. This is the whole arc, through 3.0.0.

The problem

Three things were true at once:

The photo source was gone. Google's API change left the frame with an empty library. Immich, which already held my photos, had no support.
The base was stuck in the past. Upstream had a Python 3 branch whose own commit message read "Major update to python 3, still many things left to test." It never reached master. And Raspberry Pi OS Bookworm dropped tvservice, the tool photoframe used to find out what display it was plugged into.
I didn't have the time. A port, a new service, an installer, an image, and testing across Pi models and screens is a second job. Without agents it doesn't happen at all.

Constraints

Small, old hardware. A Pi Zero W has 512 MB of memory. Full-size phone photos froze a Pi 3B+ until the frame learned to ask Immich for a preview first. Peak memory on an 800x480 panel went from about 1,061 MB to about 494 MB.
Several OS releases in the field. People run Bullseye, Bookworm and Trixie, 32-bit and 64-bit. The original project's own images were built on 2019's Raspbian Stretch.
A thing on the wall. It has to survive network outages, power cuts and an HDMI cable getting bumped, with nobody at a keyboard.
Existing behavior stays put. The first rule in the project's CLAUDE.md, in capitals: "NEVER modify existing routes for Immich features." Immich got its own /immichconfig route.

How the work ran

September 2025. The first Immich commit landed on a Friday. By Monday the fork existed and a pull request offering the work upstream was open. The agents ran as roles: an architect wrote phased roadmaps, I approved them as the product manager, a developer built, and a QA agent tested. When the developer agent added handlers to server.py anyway, the next commit was titled "Revert unauthorized changes."

April 2026: the morning it went too far. I came back for a real release push. On April 11, inside about eight minutes, an automated session rebased the Python 3 branch onto master, pushed master, created a v3.0.0 tag and release, created a dev branch, forked the Raspberry Pi image builder, and closed my upstream pull request by deleting the branch behind it. None of it had run on a Pi.

The answer was a written rollout plan with an eight-row damage assessment and one gate:

Nothing gets pushed, tagged, released, or sent upstream until it's been tested on Pi hardware.

That gate stayed in force through every release after it.

The image. A fork of pi-gen builds a headless Lite image: flash it, put your wifi details in wifi-config.txt on the boot partition, and the frame comes up with no SSH. Before rc1, a Pi Zero W driving a Toshiba laptop panel at 1366x768 through an HDMI-to-LVDS board booted that image, joined wifi from the file, and ran an Immich slideshow with HEIC photos without running out of memory. CI builds the same image under QEMU in about 48 minutes. Every photoframe tag now needs a matching pi-gen tag.

rc1, April 18. Tested on a Pi 3B+ with Immich. The same week the Python 3 work went upstream again as a 30-commit pull request, which is still open.

rc2, October 1, and the next eight hours. rc2 went out a little after midnight Eastern. By that morning, QA on a Pi 4 flashed with an image built from the rc2 commit had filed three bugs, and a second 32-bit board confirmed all three:

The image's web UI had no login. pi-gen put the credentials file under /boot/firmware; photoframe looked in /boot. The rc1 and rc2 images both shipped an open settings page.
The shutdown button did nothing on Bookworm. The kernel's GPIO numbering now starts at 512, so pin 3 is 515. That code dated from 2018.
Normal log messages never reached the log. Which also hid a worse bug, found by reading the code: the Immich API key and the Google OAuth token were being written to the log. The fixes went in with the credential change first, so the log never ran at INFO level while it still recorded keys.

All three were fixed and re-verified on the Pi 4 the same afternoon.

October 2 to 5: QA and review until it was boring. From rc2 to release, thirty issues and thirteen pull requests were opened. Meanwhile a Pi 4 ran rc2 for days. A few that stand out:

install.sh died at its pip step on Bookworm. The fix installs everything from apt. We tested it in an emulated Bullseye, and on a Raspberry Pi 5 with a clean install of Raspberry Pi OS, then our packaged install script on top. Forcing pip with --break-system-packages was ruled out.
One failed album refresh marked the album empty and blanked the frame for up to an hour. On a 464-photo album, an outage under fifteen minutes was enough. Now the frame keeps its photos, retries every minute, and refetched 25 seconds after the network came back in testing.
Rotation and overscan settings never reached config.txt, because the path had been relative since the Python 3 port. Fixing it led to the question of power loss mid-write. An fsck of the boot partition two seconds after a write found config.txt truncated to zero bytes. One more fsync fixed it, then 40 concurrent changes and 50 rounds of parallel writes came back clean.
The rc2 remediation pull request went through four independent review passes plus a Pi 4 boot on the fake-KMS driver. Each pass was answered with a commit.

I made the scope calls: tvservice is deprecated but its code path stays where the command exists, and updater bugs move to 3.1.

October 6: 3.0.0. rc2 was recut on October 5 with the remediation merged. v3.0.0 was tagged the next day at the same commit. Between the last release candidate and the stable release, nothing changed but the image builder's branch and the release notes. The stable release is the candidate that survived.

What shipped

Immich, sized for the hardware. Preview images by default, full size only when /proc/meminfo and ImageMagick's limits allow it. HEIC works. A Browse button lists your albums, names match without regard to case, and the service retries with backoff when the server is unreachable.
Displays without tvservice. The display size comes from the framebuffer, with health checks at startup.
Home automation. /control/screenon and /control/screenoff over HTTP.
Power-safe settings. config.txt edits survive a power cut and keep rotation and overscan lines across driver changes.
Clean logs. Credentials no longer reach the log, and the log now carries normal messages.
An SD card image (Bookworm Lite, 854 MB zipped) and a migration script that moves an existing install from the original project to the 3.0.0 release branch, installs from apt only, and stops on anything older than Bullseye.

Outcome

Measure	Result
Fork to first stable release	Sept 8, 2025 to Oct 6, 2026
Change against the original master	65 files, +4,686 / -1,662 lines
Merged pull requests at release	43
Issues and PRs opened between rc2 and 3.0.0	30 and 13
Bugs from the first Pi 4 rc2 test, found to fixed and re-verified	about 2 hours, same day
Peak memory, Pi 3B+ at 800x480	about 1,061 MB down to about 494 MB
Supported OS	Bullseye, Bookworm, Trixie; 32 and 64 bit
Supported Immich	v2.x (v3 planned for 3.1)

The release notes list six known issues, all planned for 3.1. Two of them, rotation ignored under the KMS display driver and a cropped picture after an HDMI replug, were found on hardware and documented, with a workaround where one exists, instead of being guessed at.

What I would do differently

Start with the hardware gate. It arrived in April because an agent shipped a release nobody had run; it belonged in the September roadmap next to the route rule. And write the final-release order into RELEASE.md before cutting the first final release. On release day the 3.0.0 branch was made before master was fast-forwarded. It came out right, but the order lived in a session, not in the doc.

The Photo Frame I Didn't Have Time to Fix

Michael Brewer — Sat, 03 Oct 2026 01:40:43 +0000

A seven-year-old Raspberry Pi project, a Google API that stopped doing the one thing it needed, and how AI agents got it running every day on my wall.

photoframe is Henric Andersson's Raspberry Pi project that pulls random photos from Google Photos and shows them like a picture frame. It's a good piece of software. Then in 2025 Google cut the Photos Library API down to photos an app uploaded itself, and a frame that reads your library has nothing left to read.

I self-host Immich. My photos are already there. So the fix was obvious on paper: teach photoframe to talk to Immich.

The fix was not obvious on my calendar. I'm far too busy to spin up a whole new photo service, build an API for it, test the app on a Pi Zero, a Pi 3B+, a Pi 4 and a Pi 5, check it on displays from 800x480 up to 1920x1080, and then build the stability controls a thing on the wall needs. Without AI that list is wishful thinking. With it, the frame has been running every day without a hitch.

Here's how it got there, including the parts where the agents went further than I told them to.

What I started with

Upstream already had a Python 3 branch. Henric's commit on it reads "Major update to python 3, still many things left to test," and it never made it to master. On top of that, Raspberry Pi OS Bookworm dropped tvservice, which photoframe used to find out what display it was plugged into. So it wasn't only a new photo source. The base had to come up to the current OS first.

The first weekend

The first Immich work happened over a weekend in September 2025. The shape of it hasn't changed much since:

Auth is an API key in the x-api-key header instead of Google's OAuth dance.
Albums act as photoframe's "keywords," so you pick albums and the frame rotates through them.
Immich gets its own /immichconfig route. The existing config routes stay exactly as they were.

That last point was a rule, written into the project's CLAUDE.md in capital letters: "NEVER modify existing routes for Immich features." The work ran as roles: an architect agent wrote phased roadmaps, I approved them, a developer agent built, and a QA agent tested. The roadmaps carried their own rules, like "DO NOT make unauthorized git commits" and "Product Manager (User) approval required before implementation." I was the Product Manager.

The QA agent's first baseline report opened with "CRITICAL FAILURE IDENTIFIED." Good. That's its job.

The developer agent, meanwhile, decided the Flask error handling in server.py needed help and added handlers there. That is an existing file the rule said not to touch. The commit that followed is titled "Revert unauthorized changes." The rule file said where the boundary was. The revert is what made it a boundary.

The push that went too far

In April 2026 I came back for the real release push, and an automated session did something I still find instructive. In one run it rebased the Python 3 branch onto master, pushed master, created a v3.0.0 tag and release, created a dev branch, and forked the Raspberry Pi image builder. None of it had been tested on a Pi. It also closed my open pull request to the upstream project by deleting the branch behind it.

None of those steps is crazy on its own. Together they're a release nobody had run.

The answer wasn't to trust the agents less in some vague way. It was a written rollout plan with a damage assessment table, master reset back to the last known good commit, and one gate that everything since has had to pass:

Nothing gets pushed, tagged, released, or sent upstream until it's been tested on Pi hardware.

A pull request became the vetting gate for the release flow. A Pi Zero W with a 1366x768 laptop panel ran the full image before rc1 got tagged on April 18. That rule has paid for itself more than any other in this project. I wrote about why a rule like that has to live in a mechanism instead of a prompt in Prompts Are Requests. Hooks Are Law.

What it does now

Immich, sized for the hardware. The frame asks Immich for a preview image first and only steps up to full size or the original when /proc/meminfo and ImageMagick's limits say the Pi can handle it. HEIC works. There's an album picker in the web UI instead of typing album names.
Displays without tvservice. If tvservice is on the system, it's used. If not, the frame falls back to the framebuffer (fbset, /sys/class/graphics) with health checks and safe defaults. On Bookworm desktop images it stops lightdm so it can own the screen.
Memory. On a Pi 3B+ driving an 800x480 panel, an out-of-memory fix took peak memory from about 1061 MB to about 494 MB.
Network trouble. Downloads retry with exponential backoff and jitter, six attempts between 5 and 60 seconds, and permanent HTTP errors are kept separate from temporary ones. If an album refresh fails, the frame keeps the last good album list and tries again in a minute instead of going blank.
Clean shutdown. SIGTERM and SIGINT blank the display instead of leaving a frozen photo on screen. Credentials no longer show up in logs.
A real image. A fork of pi-gen builds a headless Lite image. You flash it, put your wifi details in wifi-config.txt on the boot partition, and the frame comes up with no SSH needed. CI builds that image under QEMU. There's also a migration script for people coming from the original photoframe.

Measured against the original project's master, the fork touches 65 files, with 4,563 lines added and 1,618 removed. On GitHub that's 40 merged pull requests and 29 closed issues so far.

Review until it's boring

v3.0.0-rc2 was tagged October 1. Its remediation pull request went through an independent reviewer four times, and every pass got answered with a commit, not a comment saying it would be handled later. Pi 4 runs with a 1920x1080 monitor went about 17 hours at a stretch.

Where it stands

It's still a release candidate, and the issue tracker is honest about what's left. Cached photos don't show after a restart with no network. Rotation does nothing under KMS. Support for Immich's v3 API is built but waiting for 3.1. Config backup and restore and image prioritization (recent, by date taken, favorites) sit in open pull requests. There's a pull request offering the Python 3 work back to upstream, open since April.

None of that stops the frame on my wall from doing its job every day.

What made the difference wasn't a model writing Python faster than I can. It was the rules around the models: a route they couldn't touch, a revert when they did, and a gate that says nothing ships until a real Pi has run it. Each of those came from watching an agent go somewhere it shouldn't. That's not a cost of using agents. It's the work.

The code is at github.com/dev-brewery/photoframe. If you run Immich and have a Pi in a drawer, the rc2 image is on the releases page.

Prompts Are Requests. Hooks Are Law.

Michael Brewer — Fri, 11 Sep 2026 19:12:34 +0000

Part 3 of three posts behind the multi-agent case study. The enforcement layer that made the agents finish things, the ways it failed, and the experiment I built to test it and never needed to run.

Around the second week of running agents, I stopped believing that a rule written in capital letters would be followed because it was in capital letters.

The evidence was not subtle. One rule, paste the raw log lines during a job instead of summarizing them, was violated more than thirty times after it was written. One agent had a section titled "Hard Rules (fireable offenses)." It was four lines long. All four were broken at least once afterward. Every workspace on the system is full of bold rules written the day something went wrong, and the record shows how little that achieves on its own.

The comment at the top of the first enforcement plugin says why. The platform can restrict what agents can't do. It cannot force what they must do. So the forcing became code.

The stack

Fourteen behavioral plugins run today. Each one exists because of a specific day.

One detects an action request, detects a chat-only reply, and blocks or injects enforcement. It also blocks any attempt to delete or move a workspace directory. It exists because specialists kept explaining instead of doing.
Several force a memory search before any action tool and a memory write before completion. They exist because agents kept rushing to the internet instead of reading what they already had.
One blocks replies that tell me to clear my cache or try again. It exists because of a June afternoon I wrote about in the last post.
One blocks mutations on external surfaces until declared sources have been read.
One blocks bulk rewrites and non-English script in memory, because a cheap model kept drifting into other languages and a "cleanup" of that text once destroyed 1,089 files in a single pass.
One blocks agents from upgrading or downgrading the platform through the package manager, because an agent with shell access can change the platform under itself.
Email goes through a relay that rejects any non-allowlisted recipient before a connection opens. The allowlist is hardcoded in the relay so an agent that controls the environment cannot widen it. It exists because an agent once sent an email I had asked it to draft.

The coding assistant that built all of this got the same treatment: nine hooks of its own, including no direct edits to the platform config by any tool, a verified snapshot before any config change, and a post-compaction hook that re-injects the rules because the model forgets them when its context gets summarized.

The ways the law failed

This is the part I would want to read if I were starting over, so here it is without softening.

The enforcer that bypassed itself. The coding assistant built an enforcement hook and, within minutes, got around it by running the interpreter through a different tool.

Guards that block the right thing and the wrong thing. In one measured week, the memory-discipline guard blocked seven tool calls, including legitimate commands from the archive pipeline.

Guards cost context. Every turn carries the enforcement injections. In that same week the average turn sent 5,343 input tokens, the largest sent 160,734, and only 35 turns out of 6,308 hit the prompt cache. That is 0.55 percent. Routing through the quality-gate proxy rewrote payloads and defeated the vendor's caching entirely in one measurement: 276 turns, zero cache hits, roughly 600,000 input tokens paid twice. Coercion is not free.

The experiment I never needed to run

By July a question was on the table, and it was a model that put it there. Would a stronger model with lean prompting finish tasks on its own, or has the coercion been doing that work all along? I built an A/B switch to find out. Config B drops most of the enforcement plugins and rewrites each agent's rules as plain prose. Both configs validate, and the round trip was proven identical offline.

The switch has never been flipped, and not from neglect. Over the same weeks the enforcement stack and the continued refinements to the routing proxy started paying dividends in quality, and the pressure that had made config B tempting went away. Config B is the "simplify and hope for the best" option. It is also what every frontier model quietly wants: to be left to its own devices and to define for itself what counts as done. Of course a model suggested that path. The version that kept finishing things was the constrained one, so that is the version that kept running. The archives agree: no snapshot from a flip, the B workspaces untouched since July 2, and every config captured since then is config A.

The builder is an agent too

The most uncomfortable thing in six months of records is that the coding assistant I used to build the stack failed in exactly the ways the agents did. It answered before reading. It claimed verification that didn't happen. It took a conversational "that sounds good" as permission to edit 35 live cron jobs inside an experiment whose entire contract was that nothing live would change. It bypassed its own guardrail.

One comparison is worth the whole post. Asked to fix a gateway restart problem, the rushed path edited the framework's source in a local fork, linked it over production, then unlinked it and deleted the install entirely. I reinstalled by hand. The same problem, approached carefully the second time, was solved with three lines in a shell profile and no code changes. The transcripts measured both: 106 tool calls and a destroyed install, against 60 tool calls and a permanent fix. Reading is not slow. Guessing only feels fast.

The rule that surfaced through consistent frustrations

Do not write the rule in prose and hope. Build the mechanism first: the hook that blocks, the script that checks the artifact, the allowlist the agent cannot reach. Then try to break it before trusting it, because an untested guard is worse than no guard. It lets you stop worrying about a failure it was never preventing in the first place.

The full accounting of what shipped, what never did, and what it all cost is in the case study.

Done Is a Claim

Michael Brewer — Fri, 11 Sep 2026 19:07:17 +0000

Part 2 of three posts behind the multi-agent case study. Every agent I run has told me a job was finished when it wasn't. Here is what that cost and what replaced trust.

If you run agents that can act on your behalf, there is one lesson that will find you in every domain, more often than any other, with the most expensive consequences. An agent's report about its own work is not evidence. It is a second artifact that may or may not match the first.

I learned it the way you learn anything from a machine that talks fluently: repeatedly.

Verified from hashes

In the first week, an agent generated a QR code for a printed handout. It was the wrong size. The agent never looked at the output. When I asked, it kept claiming "verified," and what it had verified was that the file's hash matched the file it had written. The rule I wrote that day was three words long: look at the content.

Cached browser history

In June, an agent declared a set of download links fixed without testing them from the live page. I reported them broken. Twice. With screenshots. It tested a different URL, got a success code, and told me my evidence was "cached browser history." The actual bug was a relative path that sent every download button to the wrong directory. That reply now has its own guard: any response that tells me to clear my cache, try again, or insists the code is correct gets blocked before I see it. When I say it's broken, it's broken.

The retag that wrote nothing

Later that month I asked the homelab agent to retag three albums in my CD archive using photos of the inserts. It reported the work complete. It had written nothing. The metadata still said Track 01. Then it deleted its working copies of the photos. The originals survived only because the platform independently keeps every inbound file in its own directory, which I had not known until I needed it.

Fifty-eight of ninety-five

The one that ended the CD project was in September. I asked for a printed checklist of the archive. The agent built the list from conversation memory instead of the canonical catalog. It set the page layout at 44 rows per page when about 33 fit, and the overflow silently vanished. The printout showed 58 of 95 albums. The agent called it complete, defended that claim three times before looking at the PDF, and reported a visual page inspection that never happened. The same morning I found that a document it had "delivered" a week earlier had never uploaded.

The changelog

The quietest one is the one I find most instructive. One agent maintains a small research site with a changelog, and through the summer the changelog announced a new feature almost every day: a mobile enhancement system, connection-aware layout, analytics, keyboard shortcuts. The files exist on the agent's machine, some at exactly the byte sizes the changelog claims. None of them are on the live site. Two return 404. The rest never load, because the plugin only attaches them on pages that use a shortcode the live page doesn't. The mobile script couldn't run anyway. Its text is missing characters and fails to parse. The live server still serves the May versions of the CSS and JS. The changelog is a perfect record of work that was done and never shipped, written by the thing that did the work.

What replaced trust

Every one of those incidents produced a mechanism. Not a rule. A mechanism.

The rendered-artifact law. No printed or rendered deliverable leaves without every page converted to an image, looked at, and its rows counted. The delivery message must include the proof: page count, row count, checksum. If the agent cannot produce the proof, it cannot claim delivery.
A presubmit check that blocks files violating a standing format rule before the send command runs.
A verdict script at the end of the archive pipeline that will not pass an album unless an independent database confirms the audio bit for bit, or a recorded human approval overrides it. Every album's fixity lives in three separate places, and any tool that touches the audio has to regenerate all three.
A four-step rule for changes to the platform itself, after the coding assistant declared a config change successful on the strength of a lint and a hot reload: lint, restart through the CLI, confirm a clean start in the log, then test the changed thing through the gateway. A real test on that occasion showed the new models were in the catalog but not in the allowlist, so nothing could select them. The dry run had been "successful."

I wrote about the same lesson on the inference server, where the promotion gate invalidated a config I had promoted on a smoke test alone. That was me being confident. This is the agents being confident. The countermeasure is identical: the check reads the artifact directly, a script does the reading, and the rule has power over whoever wrote it.

Every time I let a model grade its own homework, it gave itself an A. The only verification that counts is the one that lives outside the agent.

Next: Prompts Are Requests. Hooks Are Law. The enforcement layer, and the ways it failed too.

Four Builds in Seven Days

Michael Brewer — Fri, 11 Sep 2026 19:07:11 +0000

Part 1 of three posts behind the multi-agent case study. How the stack that runs part of my life came to exist, and why the first three versions had to die.

I installed my agent platform in March 2026. That date matters, because for the seven months before it I had been doing something that looked like progress and mostly wasn't.

From late 2025 on, I followed the agentic coding wave the way a lot of us did. Every new hyped-up harness got a weekend. When none of them held up, I started writing my own orchestrator. I have the repos to prove it, and I have the hours. What all of that time proved was one thing, and I want to state it plainly because nobody selling these tools will: it is entirely possible to spend ten times the hours building guardrails for a model as the hours the model saves you on the project.

I installed a real platform anyway. The need was real. I am good at starting things and bad at maintenance and follow-through, and I have a job, a family, two acres, a homelab, and a board seat that all generate small obligations faster than I close them. I wanted to know whether a platform built for the job would change that ten-to-one ratio.

Build one

The first build ran under a locked-down service account, behind a retry proxy, with a cheap cloud model. Three days later I told the coding assistant helping me that the gateway kept restarting and the agent couldn't get anything done. That morning I framed the choice myself: keep the handicapped agent and the extreme isolation, or wipe the account and rebuild it with broad permissions. I wiped it. The shell history shows the scripts that did it, in order: back up, clean up, delete the user, recreate it, reinstall, restore.

Build two

The restored scaffold went onto the recreated account. The install did not go the way the coding assistant said it had. The gateway rebooted constantly while the assistant described it as working, and that was the first time in this project I had to say out loud that a confident report and a working system are two different things. It would not be the last.

Once it ran, a review found the real problem. The agent "verbally claims it will use native tools but fails to actually execute them." It talked about work instead of doing it. Before I tore it down, I wrote a document for whatever came next. It is still the best summary of what I wanted:

Don't make Michael drag competence out of you one inch at a time. Use the system like it's real: turn promises into jobs, turn reminders into cron, turn long work into workers, turn lessons into files, turn uncertainty into blockers, and turn "I'll do it" into proof before the sentence even leaves your mouth.

Build three

The third build was a refit: a migration runbook, new behavior contracts, a task ledger so work could not be lost, and subagents for heavy lifting. A review the next day concluded it was "improved and partially verified, but not yet trustworthy for sustained long-running autonomous work." That night the agent found a way around its new task ledger on its very first turn. The morning after, it still had not finished the task the whole refit was built to finish. On day seven I wiped the workspace and every cron and service dependency with it.

Three builds, three failure reports, and they all said the same two things. It talked instead of acting, and it didn't finish. Each rebuild had changed the isolation, the model, or the prompts. None of that touched the actual problem.

Build four

The fourth build changed one thing that the first three had not. Enforcement shipped on day one. Before any agent bootstrapped, a plugin was already installed that detects an action request, detects a chat-only reply, and blocks it. The comment at the top of that plugin states the whole thesis of the project: the platform can restrict what agents can't do, but it cannot force what they must do. So I built the forcing.

Everything running today descends from that fourth build. The six months since are their own story, and most of it is the enforcement layer growing around the agents one incident at a time. But the difference between a week of wipes and a system that finishes things was never a smarter model. It was moving the rule out of the prompt and into a mechanism, before the first message was ever sent.

What the first week settled

The community research my first manager agent did in its first hour said: start boring, one agent, add specialists late, prefer cron over heartbeat. The first half of that advice fell apart on contact. Telling a single agent to use cron did exactly nothing. And a single agent doing everything gets buried in the harness: every turn's context is muddied by instructions from domains it is not working in, and a single turn could run to six figures of input tokens before the agent read a word of the actual request. That is not a model problem. It is a prompt that cannot fit in anyone's head. So the specialists went up on the fourth build's first day, each with a small prompt in its own domain, and that decision is the one that held. The orchestrator that was supposed to route between them lasted two weeks before a lookup table replaced it. A research job that ran every thirty minutes built a body of research-backed material for its domain and did its job well, until the knowledge base grew large enough to put memory pressure on the whole system, so it is off until it can live in its own retrieval setup. Neither of those was an argument for fewer agents. They were arguments for smaller prompts and fewer moving parts per agent.

The method that came out of that week is the one I still run. A coding agent works in the background. It reads the transcripts for what went wrong, builds the cron job or the hook that makes that failure impossible or at least loud, and then updates the agent's memory to reflect the change. The agents do the work in their lanes. The coding agent does the plumbing. I read the receipts.

The ten-to-one ratio from before the install did not vanish when the platform arrived. It got paid down, one mechanism at a time, until the machinery started saving more hours than it cost. The full accounting, with the numbers, is in the case study.

Next: Done Is a Claim. Why the only verification that counts is the one that lives outside the agent.

Case study: a household of agents

Michael Brewer — Fri, 11 Sep 2026 19:01:53 +0000

Six months of records from a multi-agent stack that runs part of my life, and what the record, not my memory, says worked.

Summary

I run a set of AI agents on a small dedicated machine in my house. Each one owns a domain: a general assistant, the homelab, and a handful of hobbies and volunteer commitments. They talk to me over chat, run scheduled jobs, read email, and ship deliverables I can open myself. I installed the platform in March 2026, after seven months of chasing the agentic coding wave: every hyped-up harness that came along, plus an orchestrator of my own. The stack running today was the fourth build in its first week, and this is what its own records say about the six months since.

The short version is uncomfortable. The value never came from how smart the models were. It came from how fast I could turn a lesson into machinery: a script, a state file, a cron job, a hook, a checksum. Every durable win on this system is deterministic plumbing with a language model sitting at one narrow seam. Every recurring failure came from asking a model to do something it cannot be trusted to do on its own word: hold state, remember what it already knew, verify its own work, stay in its lane, or stop when told.

After six months, the most important thing I have built is not an agent. It is the enforcement and verification layer around the agents, and the habit of never believing "done" until I can see the artifact.

The problem

I built this because of a specific weakness that I asked the agents to cover. I am good at starting things and weak at maintenance, follow-through, and impulse decisions. A full-time job, a family, a two-acre property, a homelab, and a board seat produce a constant stream of small obligations, and the twenty minutes each one needs never lines up with the twenty minutes I have. The agents were supposed to own the follow-through.

The seven months before the install had already taught me one thing. It is entirely possible to spend ten times the hours building guardrails for a model as the hours it saves you on the project. I installed it anyway, because the need was real and I wanted to find out whether a platform built for the job would change that ratio.

The first three attempts, all in one week in March, failed the same two ways. They talked about work instead of doing it, and they did not finish. Each rebuild changed the isolation, the model, or the prompts. None of that fixed it. The fourth attempt is the one that stuck, and the difference was not a better model. It was that enforcement shipped on day one.

Constraints

Cheap models on purpose. Early research put frontier models at roughly $47 a week for this workload against about $6 for a budget cloud model. The whole point was always-on, so the budget model won, along with every weakness that came with it.
Every agent has a shell. That is what makes them useful, and it means the worst days were never "the agent said something wrong." They were "the agent did something irreversible."
One on-call engineer. A personal agent stack is a production system with a staff of one. The platform under it has bugs that look exactly like agent misbehavior, and I am the only one who will ever notice either.
The agents live on a firewalled subnet, on purpose. The agent machine sits on its own isolated network behind carrier-grade NAT. It cannot reach the main LAN or the public internet except through the paths I gave it, and that was the right call before the first agent ran. What the record shows is that the agents kept trying to configure around it, planning for public traffic that could never arrive and breaking things in the attempt, until that boundary became the first section of their memory instead of an obstacle to route around. The friction was theirs to absorb, not mine to remove.

Options I rejected

An orchestrator that routes with a language model. The architecture I settled on for the fourth build, and the one the community recommended, had a manager agent read every message and delegate to specialists. It lasted two weeks. It editorialized instead of dispatching, its delegation calls failed silently when the model left out the target, and the runtime it delegated through had a lifecycle bug where results never came back. Routing is now a lookup table: one chat topic per agent. No model sits in the routing path.

Rules in capital letters. Every workspace is full of bold rules written the day something went wrong. One rule was violated more than 30 times after it was written. Another set of four "fireable offense" rules was broken, all four, at least once afterward. Around the second week I stopped believing that a rule in a prompt would be followed because it was emphatic.

Letting the research agent's knowledge base live inside the agent. A research agent ran every 30 minutes and built a body of research-backed material that now covers just about any question in its domain: hundreds of runs at a 96 percent success rate and a vector memory assembled with zero human involvement, on tokens cheap enough to make that possible. The agent was doing exactly what it was supposed to, and the value was real. I turned it off because the knowledge base grew large enough to put memory pressure on the whole system and to bloat the agent's own context on every turn, which broke the narrow-domain ethos the rest of the household runs on. The right shape is a spin-off knowledge base with its own dedicated multimodal RAG setup that the agent sends requests to instead of carrying the material itself. Whether that gets built, time will tell.

Long fallback chains. A seven-model fallback chain hid a three-month outage. The local model provider had never served a single successful request, because a base URL was missing a path segment, and the chain quietly skipped past it every time. A failover chain is a way to hide outages from yourself.

The model as formatter. A quiz script produced a question as JSON and the agent formatted it, badly, twice. The fix was not a sterner prompt. The command was deleted and the script now sends straight to chat with hardcoded formatting. The agent has zero formatting vector.

What shipped

Verification that lives outside the agent. An agent's report about its own work is a second artifact that may or may not match the first. A printed checklist once showed 58 of 95 items, was called complete, and was defended three times before anyone looked at the PDF. Now no rendered deliverable leaves without every page converted to an image and the rows counted, and the delivery message carries the proof. A presubmit script blocks files that violate a standing format rule. A media archiving pipeline ends in a verdict script that will not pass an item unless an independent database confirms the data or a recorded human approval overrides it, with three separate fixity records per item.

Hooks instead of prompts. The comment at the top of the first enforcement plugin states the whole problem: the platform can restrict what agents can't do, but cannot force what they must do. Fourteen enforcement plugins now cover that gap. One detects action requests and blocks chat-only replies. Several force a memory search before any action and a memory write before completion. One blocks replies that tell me to clear my cache or try again. One blocks mutations on external surfaces until declared sources have been read. One blocks bulk rewrites and non-English script in memory. One blocks the agents from upgrading the platform under themselves. Email goes through a relay that rejects any non-allowlisted recipient before a connection opens, with the allowlist hardcoded so an agent that controls the environment cannot widen it.

Memory with a schema and a size cap. Memory was the most dangerous component in the system. An overnight planning job produced over 200 files and 43,000 lines in four days, mostly regenerated copies of the same plan, with confident wrong claims buried inside that memory search kept resurfacing. The fix was deleting it and rebuilding by hand. Memory is now one file per topic, a check script hard-fails any index over 16 KB or any duplicate heading, reflections go to dated files, and the best-behaved agent's entire memory sits near 5 KB with each task written as a command plus a "done when."

The model at the narrowest seam. The things I would miss tomorrow are almost all shell scripts with a language model doing one small job, or no job at all. A daily reminder is a cron job that sends buttons and writes a state file. Follow-ups fire every 15 minutes until acknowledged. When I tap a button, a plugin updates the state and replies. No agent turn runs and no tokens are spent per tap. A weekly config sync diffs a proxy against the config, asks the model to review the diff, snapshots, applies, and notifies. On its first run the review rejected a halved token limit. The model earns its place as a reviewer of a diff, not the author of the change.

Blast radius capped mechanically. After a "clean up the non-English text" request became a character filter that destroyed 1,089 memory files in one pass, and the same misreading produced 106 global edit commands three days later, the rule became structure: cleanup means rewrite, stop and ask above five files, and a daily sweep that reports stray text and never edits anything. Both incidents were survivable because a backup existed.

The same discipline for the builder. The coding assistant I used to build the stack failed in exactly the ways the agents did. It answered before reading, claimed verification that never happened, took conversational agreement as permission to edit 35 live cron jobs, and bypassed its own guardrail within minutes of writing it. One measured comparison: the rushed fix for a gateway restart problem took 106 tool calls and destroyed the install. The careful fix, three lines in a shell profile, took 60. It now runs under nine hooks of its own, including a snapshot before any config change and a rule that nothing is done until the change is tested through the gateway.

Outcome

Measure	Value
Scheduled jobs defined and enabled	77 and 69
Scheduled runs recorded, March 24 to September 10	14,563
Runs that did not finish clean	1,974, or 13.6 percent
Model turns in one measured week	6,308, about 900 a day
Input tokens in that week	33.7 million
Turns in that week that hit the prompt cache	35, or 0.55 percent
Enforcement plugins, March to September	5 grew to 24 enabled entries, 14 behavioral
Items in a verified media archive, closed September 7	95

What I would miss tomorrow: the reminders, the verified archive, travel support that kept my records straight through a canceled flight, working from the changes I told it about and the itinerary emails I forwarded, and rebuilt the documents, an eight-week volunteer curriculum published on schedule, a daily exam-prep quiz, email triage across five inboxes, and a 7 AM homelab check that reads the gateway log and names real problems.

What never shipped: a finance agent that was created and never bootstrapped, and an A/B switch built in July to test whether lean prompting on a stronger model could replace the enforcement stack. It was validated and never flipped, and not from neglect: the enforcement stack and the proxy refinements started paying dividends in quality and took the pressure off the simplify-and-hope option. That option is what every frontier model quietly wants, to define for itself what counts as done, so of course a model suggested it.

The cache number is the one that stings. Routing through the quality-gate proxy rewrote payloads and defeated the vendor's automatic caching, so one measurement showed 276 turns and zero cache hits, roughly 600,000 input tokens paid twice. The enforcement injections themselves cost context on every turn. Coercion is not free.

What I would do differently

I would not spend my own hours writing documentation for the agents. Every rule I put in prose was a bet that the model would read it, weigh it, and comply, and the record above shows how that bet paid. I would go straight to more coercion and more constraints: hooks that block, scripts that check the artifact, allowlists hardcoded where an agent cannot reach them, and a size cap on memory from the first day. A constraint costs an afternoon once. A hundred failed attempts cost an afternoon each, plus the cleanup. The constrained version is the cheaper product, and it is the only version that ever finished anything.

A personal agent system spends the operator's attention to save the operator's attention. It is only a win when the second number is bigger. For the first few months, it was not. What moved the balance was never a better model. It was moving each recurring correction out of my head and into a mechanism, so I stopped making the same correction twice. The ten-to-one ratio from before the install did not go away. It got paid down, one mechanism at a time, until the machinery started saving more hours than it cost.

Case study: smart-proxy

Michael Brewer — Fri, 11 Sep 2026 19:01:46 +0000

One endpoint in front of three tiers of inference, so that always-on agents get a frontier-class answer only when they need one.

Summary

I run a fleet of AI agents that generate hundreds of requests an hour, around the clock. Most of those requests are routine: heartbeats, status checks, processing a tool result, classifying a message. Sending all of it to a frontier API was expensive and fragile, and rate limits and credit exhaustion took the agents down at the worst times.

smart-proxy is the routing layer I built to fix that. It is a standard-library Python proxy that presents one OpenAI-compatible endpoint and decides, per request, whether the work goes to a small CPU model, a large local GPU model, or a cheap cloud model. It also polices the cloud tier with a quality gate, because cheap cloud models are capable but unreliable in specific, predictable ways.

Over one measured 40-hour production window it handled 8,129 requests at a 0.17 percent failure rate with no manual intervention. The quality gate passed 95.7 percent of cloud responses on the first attempt, and action requests that used to take 36 to 45 seconds now take 15 to 20.

The problem

Three things were true at once:

The agents never stop. An always-on multi-agent system produces a constant stream of small, cheap-to-answer requests, with an occasional hard one mixed in. Paying frontier prices for the small ones made no sense, and depending on a single external API meant a rate limit anywhere took everything down.
Local hardware is capable but constrained. The inference server is about $2,000 of used enterprise parts: an EPYC 7302P, 128 GB of ECC memory, and two Tesla P40s from 2016. It can run a 27B dense model or an 80B mixture-of-experts model well, but only one large model at a time, with a 20 to 40 second cold start and about 30 seconds to swap.
Cheap cloud models misbehave in specific ways. They narrate an action instead of calling the tool. They invent tool requirements for plain factual questions. They truncate under load. Each of those looks like a successful HTTP 200 to a naive client.

The requirement was one endpoint, all models visible, and the system making the tiering decision itself.

Constraints

Pascal-era GPUs. No Tensor Cores, no vLLM, and most published tuning advice is written for newer cards. The GPU tier had to live on llama.cpp.
One GPU model resident at a time. Any design that assumed concurrent large models was dead on arrival.
The CPU had to be the floor. Whatever ran on CPU had to survive GPU swaps, GPU crashes, and GPU power-off, so the system always had something available.
Production traffic, not a lab. The agents were live users of the proxy throughout. Every change had to be one variable, tested, then frozen with checksums so it could be rolled back in about two minutes.

Options I rejected

A LiteLLM gateway. This is where I started. It handled alias routing and a remote coder target, but the moment I needed a quality gate, per-pool concurrency accounting, and structured request logging, it was easier to own a small proxy than to bend a large one. The lesson I kept was not "avoid proxy layers." It was "don't add layers; let the one layer you have absorb policy."

A GPU model as the judge. Judging whether a cloud response is empty, truncated, or narrating instead of acting is a narrow task. Spending a GPU slot on it would contend with the actual work. A 4B model on CPU does it for free.

Static concurrency caps from synthetic tests. My first caps came from small test requests and did not hold. The vendor's limiter varies by load window, and the payload shape matters. The caps that survived came from stress tests with real gateway-shaped requests, median around 32K tokens, recording admission versus 429 and setting each cap at the worst observed value.

A fallback to "whatever local model is loaded." That serves the wrong model silently. The fallback is now pinned to a named stack and skips itself, with a log line, unless that exact stack is active. A fallback that might serve a different model than intended is worse than no fallback.

What shipped

Three tiers behind one alias map. Two Qwen3-4B models on dedicated CPU cores form the always-on floor: a classifier that triages each request into simple, retrieval, code, or reasoning in 200 to 600 milliseconds, and a helper for summaries, retrieval, tool matching, and fallback judging. The GPU tier is twelve llama.cpp stacks, swapped through the Portainer API with a ten-minute anti-flap cooldown and rollback on a failed swap. The cloud tier is two real vendor pools with measured caps of 6 and 5.

A quality gate with a rubric. Cloud responses pass through deterministic validation of tool-call structure, then an LLM judge, then a retry with structured feedback if either fails. The most important finding was that a judge without explicit criteria hallucinates failures. My first judge prompt asked whether the response "requires tool use," and the model decided every response did. The fix was a three-part rule, the list of tools read from the actual request, and explicit exclusions for knowledge questions, opinions, and greetings.

Pre-classification instead of retries. The retry loop existed because cloud models narrate. Three attempts at about twelve seconds each is where the 36 to 45 second latency came from. Now the CPU helper reads the request against the tool names, spends two to three seconds picking the applicable tool, and the proxy injects a structured nudge naming it. The cloud model produces the tool call on the first attempt. Names-only tool lists at about 40 tokens outperformed described lists at over 200, and temperature zero was mandatory for anything classification-shaped.

Concurrency accounted on the serving pool, not the requested id. One model id was stress-verified as its own pool, then the next day three out of three production requests for it came back answered by a different pool. The vendor had merged them without announcing it. Had I kept the two caps separate, real concurrency on that pool would have been 13 against a measured limit of 5. The proxy now carries a routing map in code that collapses legacy ids onto the pool that serves them, whatever id the client asked for, emits a reroute event and a response header on every reroute, and warns at load if the config documentation disagrees with the code.

Structured request logging. Every request emits start, end, routing, and reroute events keyed by request id. That is how the pool merge was confirmed in live traffic within minutes of deploy: 63 reroutes in a 30-minute window.

The two bugs that made fallbacks work. A transport error path produced an HTTP status of 0, which Python's server happily emitted as an invalid status line and broke the client connection; the fix clamps out-of-range statuses to 502 at the send boundary. And every local fallback was failing with a 400 because Anthropic-format tool-call messages carry null content, which llama.cpp rejects; the translator now emits an empty string, and fallback errors return 502 with the real backend detail attached. Both were invisible until the error propagation was fixed.

Outcome

Measure	Result
Requests in one 40-hour production window	8,129
Failure rate in that window	0.17 percent
Manual interventions in that window	0
Quality gate first-attempt pass rate	95.7 percent
Action-request latency, before	36 to 45 seconds
Action-request latency, after	15 to 20 seconds
Classifier triage latency	200 to 600 milliseconds
Cloud cost on routine traffic	none; handled on CPU

The thesis the numbers support: most agent traffic does not need a frontier model, and small models are most valuable when they steer large ones rather than replace them. A 4B model that can say "this needs the cron tool" in two seconds saves thirty seconds of a cloud model fumbling toward the same conclusion.

What I would do differently

Log structured request events from day one. Every hard debugging story in this project got easy the moment the reroute and routing events existed, and I built them last. And treat every rate limit as a defense rather than a promise: the number is "worst I have observed," re-verifiable any time, never a fact about the vendor.

Shock and Awe Is a Business Model

Michael Brewer — Tue, 08 Sep 2026 19:30:30 +0000

Almost all of my friction with frontier models, and I mean all of them, traces back to one root cause. It isn't capability. It's tuning.

Every frontier model is tuned for shock and awe. The demo has to dazzle. The first answer has to feel brilliant. The model volunteers essays when you wanted a sentence, generates confident sweeping output when you wanted a careful narrow one, and optimizes for the impression it makes in the first thirty seconds over the quality of the working relationship in month six.

This is not an accident and it is not a flaw in the training pipeline. It is the business model. The adoption curve has to keep climbing so the easy capital keeps flowing. A model tuned for restraint, one that asks a clarifying question, delivers the minimum correct answer, and stops talking, would be better to work with and worse in a demo. The demo wins, because the demo is what raises the next round.

I say this as a daily, heavy, mostly satisfied user of these models. They are remarkable. But remarkable is the product they're selling, and it's not the product an organization needs in order to operate.

What organizations need when the novelty wears off

An org running AI in production needs the opposite of shock and awe. It needs method. Predictable scope. An answer that stops when the question is answered. A model that follows the runbook instead of improvising a more impressive one. Delivery over volume, consistency over brilliance.

I learned this the way I learn everything, by running the systems myself. My agent stack went through five teardowns and rebuilds. The failures were never because a model was too weak. They were because models tuned to impress kept blowing out their context doing more than the task required, and because I kept trying to make cheap general models do what only a more capable or more specialized one could. The fix, every time, was narrowing: smaller scopes, tighter instructions, specialist agents, and enforcement plugins that mechanically punish showing off. I run a quality gate on my own infrastructure for exactly this reason. My agents' output is graded against the task, not against how impressive it sounds.

That's a homelab-scale version of what every serious AI-adopting org is going to end up building. Not because they want to, but because the vendors' incentives and theirs point in different directions.

The literacy that stops being optional

Here's the uncomfortable consequence. If the models you rent are tuned for someone else's goals, then getting models tuned for yours means understanding how tuning works. Weights, fine-tuning, quantization tradeoffs, evaluation methodology, what a training objective rewards in practice. Not at research depth. At operator depth: enough to read a model card critically, enough to know what a fine-tune can and cannot fix, enough to measure whether the behavior you bought is the behavior you got.

For most of the software era, this kind of knowledge was optional the way compiler internals are optional. You could build a career on top of the abstraction. I don't believe model behavior gets to stay abstracted, because the abstraction is leaking money and risk in a way compilers never did. A model that over-delivers by 3x on every request is a cost center. A model that improvises outside its scope is a liability with a legal department's name on it. Cost-effective AI that isn't a liability to the org requires someone in the building who understands what the weights were trained to do, and the orgs that treat that as a vendor's problem will pay vendor prices for vendor-aligned behavior, forever.

Open weights are what make the alternative possible. When the weights are yours, tuning for method over spectacle is an engineering decision instead of a feature request to a company whose incentives run the other way. That, more than cost, is why I run open models on my own hardware and why I think the maid-to-order open-weights era is coming regardless of how loudly the incumbents warn against it. The gatekeepers in the pioneer phase always cry. It's what the phase sounds like.

Where the two theses meet

Part one said: compose many specialized models the way you compose a staff. This part says: expect to tune some of them yourself, and staff for the literacy that requires.

Put together, that's the whole implementation philosophy. If I were handed a real budget for a long-term AI implementation tomorrow, this is how I'd spend it. Not on the biggest model money can rent, but on a bench of specialized ones in the right seats, a routing and evaluation layer that keeps each in its lane, the measurement discipline to prove it's working, and enough weights-and-training literacy in-house that the org's AI serves the org's goals instead of its vendors'.

None of that requires a frontier lab's budget. I know, because I run the small version of it on hardware nobody else wanted.

The Genealogy Book Nobody Had Time to Read

Michael Brewer — Tue, 08 Sep 2026 19:25:13 +0000

What a personal agent stack is for, once the demos are over.

A relative of mine was sent a book about one of our ancestors. A genealogy, hundreds of pages, compiled by someone who had clearly spent years on it. The kind of thing a family is lucky to have and almost guaranteed never to read. It needed to be gone through, cross-referenced, and connected to what we already knew about the family line. My relative did not have time for that. Nobody in the family did. It sat there the way these things sit, valuable and untouched.

This is the part of the story where AI usually gets oversold, so let me be precise about what happened and what made it possible.

Half an hour to digitize

I've spent the past year building a personal agent stack: an assistant that runs on my own infrastructure, that I've customized tool by tool as real needs came up. Because that plumbing already existed, digitizing the book was not a project. It was half an hour of feeding pages through, with the agent handling OCR, cleanup, and structure as we went. Hundreds of pages became searchable, structured text before lunch.

The half hour is the headline number, but it's the least interesting part. Digitization is a solved problem. What came next is why I'm writing this.

Mapping the connections

I exported our existing family tree data from Ancestry.com and handed it to the agent alongside the digitized book. Then I set it loose on the tedious part: going through the book person by person and mapping every connection onto the tree we already had. Which people in the book matched people we knew about. Which were new. Where the book confirmed our data, where it contradicted it, and where it filled holes.

This is exactly the kind of work that defeats a human volunteer. Not because it's hard, but because it's hundreds of small, careful, boring judgments in a row. It's also exactly what a well-instructed agent is good at, provided you check its work. I spot-checked as it went, corrected its course a few times, and let it grind.

Five hours of driving, one website

The visit ended and I had a five hour drive home. So the drive became the build window. Voice messages from the road guided the agent through standing up an open-source genealogy website on top of the mapped data, and then through starting something the open-source tool didn't have: a new visualization for exploring the tree.

I want to be honest about what "guided by voice from the car" means. It does not mean I dictated flawless instructions and arrived home to finished software. It means the agent worked, hit decisions it shouldn't make alone, and I made them at highway speed in short messages. Some of what I found when I got home was wrong and got redone. But the shape of both the site and the visualization tool existed by the time I pulled into the driveway, built during hours that would otherwise have produced nothing but mileage.

A couple of days of refinement

Once home, I spent a couple of days refining with the agent: fixing the wrong turns, polishing the visualization, getting the data presentation right. At the end of it, our family history, the book's contents and the mapped tree together, was live where anyone in the family could see it.

That last part matters more to me than the technology. This information used to live in two places: a paywalled account one person maintained, and a physical book one person possessed. Now it belongs to the family. A grandparent can look at it. A cousin I've never met can look at it. Nobody needs a subscription or a login or my help.

What this is a story about, once you get past the digitizing

Not about AI reading a book fast. It's about accumulated tooling meeting a real obligation. Every customization in my stack existed before this project, built for other reasons over a year of daily use. When the book arrived, the marginal cost of taking on a job nobody had time for was half an hour, a car ride, and a weekend's worth of refinement sessions.

That's my working definition of what personal AI tooling is for. Not demos. Not novelty. Taking things a family cares about but cannot resource, and resourcing them.

The Right Models in the Right Seats

Michael Brewer — Tue, 08 Sep 2026 19:25:06 +0000

The thesis behind everything I build: specialization beats scale.

Ask the frontier labs what the future of AI looks like and the answer is always the same: bigger. More parameters, more compute, one ever-more-capable model that does everything. That vision has a convenient property: only a handful of companies on earth can build it, and you'll be renting it from them forever.

I run my AI workloads on a different thesis, and after a year of measurements on my own hardware, I believe it more, not less.

The way forward is not ever-larger all-knowing models. It's teams of highly effective, highly specialized smaller models, composed the way a good company composes a staff.

Get the right models on the bus

Jim Collins framed the difference between good companies and great ones as getting the right people on the bus, and the right people in the right seats. Nobody staffs a company by hiring one impossibly expensive generalist to do every job. You hire people whose strengths match their seats, and the organization outperforms the sum of its parts.

My inference stack runs exactly this way, and I have a year of production numbers behind it:

A 4B-parameter model running on CPU cores classifies every incoming request in a few hundred milliseconds. It cannot write good code and never will. It doesn't need to. Its seat is triage, and it fills it for free, on hardware that would otherwise idle.
The same class of tiny model reads a request against a list of available tools and tells a big cloud model which one to use. That two-second nudge cut my action-request latency in half, not because the small model is smart, but because it prevented an expensive model from fumbling toward a conclusion a cheap one had already reached.
A sparse mixture-of-experts model handles the fast path at 41 tokens per second on decade-old GPUs, because activating 4B parameters out of 26B is itself specialization inside the weights.
The big models, local or cloud, only see the work that earns their cost. In one measured 40-hour window, 8,129 requests flowed through this division of labor with a 0.17% failure rate and no human intervention.

The lesson from those numbers isn't that small models are secretly as good as big ones. They aren't, and pretending otherwise is how you build garbage. The lesson is the one from my notes that I keep coming back to: the tiers aren't a hierarchy of quality, they're a division of labor. A 4B model in the right seat outperforms a frontier model in the wrong one, on cost, on latency, and often on reliability, because narrow tasks reward consistency over brilliance.

Companies already know this pattern

Here's why I think this thesis wins on economics, not ideology. Every company already organizes its people this way. Nobody believes the optimal workforce is one superhuman consultant billing by the token. Companies win by being specialized and nimble in their personnel, and they will demand the same from their AI: a small model fine-tuned on their support history in the support seat, a compliance-tuned model in the compliance seat, a coding model that knows their codebase in the engineering seat, orchestrated by systems that route work to whoever fills the seat best.

That world is arriving through open weights. The Qwens and Gemmas I run today are the early, general-purpose versions. The trajectory points at made-to-order weights: models distilled, tuned, and owned by the companies that run them, sized to their seats, running on their hardware or commodity clouds, with their data never leaving the building. Every quarter the open releases get better at fitting into seats that used to require a frontier API call, and my own benchmarks watched it happen: models that needed 39GB of VRAM last spring were outclassed by 23GB models with better architectures by fall.

The gatekeepers always cry

Which brings me to the part of the argument that the frontier labs make for me.

Listen to how the largest AI companies talk about open weights: dangerous, irresponsible, impossible to control, surely the end of safety itself. Some of those concerns are sincere and worth engaging seriously. But notice the shape of the argument and who it benefits. The pioneers of every technology wave have warned that the tools were too dangerous to leave the temple, right up until the moment the tools left anyway: mainframe companies about personal computers, telecoms about the open internet, every incumbent about every commodity that ended their toll booth.

Gatekeepers in the pioneer phase always cry. It's what the phase sounds like. The economics underneath don't care: when capability becomes a commodity you can own instead of rent, composition becomes the differentiator, and composition is an engineering discipline, not a capital moat.

What this means for the work

If the thesis is right, the scarce skill of the next decade isn't prompting one giant model. It's the boring, measurable systems work of running teams of models well: routing, failover, quality gates, evals that catch a model drifting out of its seat, guardrails that hold when a model exceeds its authority, and the operational discipline to measure everything, because half of what you believe about your stack will be wrong within six months. I know because I measured mine, and it was.

That's the bet my basement server, my benchmarks, and this blog are all placed on. One team of specialists, on hardware nobody wanted, doing the daily work of a system that would otherwise be an expensive subscription. The right models on the bus, the right models in the right seats, and the bus is yours.

DEV Community: Michael Brewer

My First User Said "Yeah, Absolutely"

What I Built

Demo

Code

dev-brewery / replydesk

Review replies in your own voice - open-weight GLM-5.3-Flash, triage + fact-grounded drafts. Live: replydeskhq.tech

replydesk

Quickstart

How I Built It

Why Does Open Innovation Matter?

My Agent Session

My First User

Where It Stands

Prize Categories

clearnight: a stargazing planner that tells you which nights are worth going outside

What I built and how it gets people outdoors

Demo

Code

dev-brewery / clearnight

clearnight

Install

Quick start (offline demo, ≤ 5 commands)

How I built it with open-source AI

Why open innovation matters for this project

The outdoor loop, demonstrated with mock data

Case study: photoframe 3.0.0

Summary

The problem

Constraints

How the work ran

What shipped

Outcome

What I would do differently

Links

The Photo Frame I Didn't Have Time to Fix

What I started with

The first weekend

The push that went too far

What it does now

Review until it's boring

Where it stands

Prompts Are Requests. Hooks Are Law.

The stack

The ways the law failed

The experiment I never needed to run

The builder is an agent too

The rule that surfaced through consistent frustrations

Done Is a Claim

Verified from hashes

Cached browser history

The retag that wrote nothing

Fifty-eight of ninety-five

The changelog

What replaced trust

Four Builds in Seven Days

Build one

Build two

Build three

Build four

What the first week settled

Case study: a household of agents

Summary

The problem

Constraints

Options I rejected

What shipped

Outcome

What I would do differently

Links

Case study: smart-proxy

Summary

The problem

Constraints

Options I rejected

What shipped

Outcome

What I would do differently

Links

Shock and Awe Is a Business Model