DEV Community: Christian Anderson

My homelab agent can only recommend what the dataset can prove

Christian Anderson — Fri, 25 Sep 2026 10:59:42 +0000

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

Toolsmith is a weekly agent that asks one question about my self-hosted homelab: is there a better tool than the one we run, and is that actually true? It measures what's running, researches replacements, tests up to two in a sandbox with no network, and writes recommendations. A validator then withholds any recommendation whose numbers aren't in the facts it cites.

After two weekly runs it had written 8 recommendations and published 3. The problem was that the validator worked on prose. It pulled digits out of sentences, so Apache-2.0 became the number 2.0, and it held back correct claims along with wrong ones.

For this challenge I moved Toolsmith's output into Sanity and built an agent that answers questions about it over Sanity Context MCP:

Structured content is the evidence. Every fact is a document with typed measurements[]{value, unit}. Every number a recommendation relies on is a numbersUsed[] entry holding a reference to the fact that must contain it. Licences are an SPDX field, not a phrase.
A Knowledge Base holds the recall. Each run's full report and the validator's "withheld" file are report documents, indexed as a Context Knowledge Base, so the agent can explain why something was held back in the run's own words.
Only the structure can make a claim "shown". The model can explore both freely, but it can present a recommendation only through one tool. That tool fetches the recommendation with its facts joined in, through Context MCP, and checks every number against the referenced fact. A number that turns up only in the prose backs nothing.

The agent only works because the content is structured. Ask it "should we replace our reverse proxy?" and it has to say that all three recommendations were withheld, not paraphrase them into something that sounds confident.

Demo

"What did the observability runs find?" Five recommendations exist for that area. Four are withheld and one (Grafana OSS) is shown. The answer guard also caught six numbers that no shown recommendation holds, and made the model take them out:

The shown recommendation claims the target container has 4440 MB free. Here is the join the validator runs, in Vision. The claim's number points at fact F037, and F037's measurements contain 4440:

"Why were the reverse-proxy recommendations withheld?" This one uses both endpoints. The agent reads the outline of the Knowledge Base, pulls the reverse_proxy and fact_verification entries through knowledge_base_read, and then presents each recommendation through the validator. The Knowledge Base prose is full of figures. The guard strips every one of them, because recall isn't evidence. What's left is an answer built entirely from the validator's structured result:

The Studio, for anyone who wants to see the shape of the data:

Code

https://github.com/casareanderson/toolsmith (MIT). The Sanity work is in sanity/:

convert/convert.py and convert/reports.py: Toolsmith run artefacts → NDJSON. Deterministic (stable _ids), stdlib only, and sanitised, because the dataset is public.
studio/: the schema. It's deployed because Context only serves datasets that have a deployed Studio (more on that below).
agent/mcp_client.py: a ~130-line Streamable HTTP MCP client, stdlib only.
agent/validator.py: validate(), which works on the joined document instead of prose.
agent/agent.py: the loop, the present_recommendation tool and the answer guard. It works with any OpenAI-compatible model. I used qwen/qwen3.7-flash on OpenRouter.

python -m agent --trace "What did the observability runs find?"   # via Context MCP
python -m agent --offline "..."                                   # public GROQ API, no token needed

How I Used Sanity

1. The schema is the validator's contract. The original validator asked "does this number appear in the text of fact F037?" The new one asks "does the fact this numbersUsed entry references have this value in its measurements?" That's one GROQ projection:

*[_id == $id][0]{
  title, verdict, claim,
  numbersUsed[]{ value, unit, "fact": fact->{ factId, measurements } },
  citedFacts[]->{ factId, licenceSpdx }
}

A licence is licenceSpdx: "Apache-2.0", so it can't be mistaken for the number 2.0. That was the false positive that started all this.

2. Two Context MCP endpoints, two jobs. An endpoint serves either GROQ tools or Knowledge Base tools. If you attach both source types, the dataset wins and the Knowledge Base is silently ignored. So there are two endpoints. toolsmith runs in GROQ mode over the whole dataset, and the validator's joins go through it. toolsmith-kb runs in Knowledge Base mode over the report documents. The agent connects to both and namespaces the KB tools as kb_*.

3. The Knowledge Base is deliberately not evidence. This is the design decision I'd defend hardest. Prose is exactly what a Knowledge Base is good at: the reports are long, and the reasons for withholding are narrative. But if a number in a report could back a claim, the validator would be back to trusting prose. So the answer guard only accepts numbers from a recommendation that present_recommendation has returned as shown. Anything else gets sent back to the model once, and after that it's replaced with [not verified].

4. What caught me out:

-32004 Only datasets with deployed Studio applications are supported. My first MCP handshake failed with this. Context needs a deployed Studio for the dataset, even if nobody ever opens it. One sanity deploy fixed it.
The endpoint name is permanent, and a GROQ filter that matches nothing looks exactly like a broken connection. The toolsmith endpoint uses an empty filter, so it serves the whole dataset.
Context wants an organisation token, not a project token. The client says so when it gets a 401 or 403.
The step limit. Each present_recommendation call is one model round. An area with five recommendations used up most of a 10-round budget before the model reached the one that passes. The limit is now 16.
My own redaction broke run IDs. The answer guard replaced an unbacked 2305 inside run-20260914-2305, so the demo printed run-20260914-[not verified]. Redaction now skips the same ID, date and URL spans the detector skips, and there's a test for it.
The Knowledge Base isn't a copy of your documents. From 4 report documents, the build produced 5 entries organised by topic (observability, reverse_proxy, fact_verification, candidates, recommendations), each with an outline of what it covers and what it leaves out. The agent navigates that outline and doesn't search raw documents.
The free plan caps a Knowledge Base source at 150 documents, and editing a filter doesn't re-index. My first source pointed at the whole dataset (275 documents), so Build entries stayed greyed out. Saving a _type == "report" filter on the existing source changed nothing. I had to remove the source and add it again with the filter (4 documents) before it would build.
Withheld reasons contained the very numbers the guard forbids. "The claim states 119.1 but no cited fact backs it" came out as "the claim states [not verified]…". Now present_recommendation gives reasons in words ("six figures in the claim have no structured source"), and counts are spelled out because a digit would itself trip the guard.
When a Knowledge Base is connected, prose goes to it. The GROQ endpoint serves the whole dataset, reports included, so at first the model just read report bodies over GROQ and never touched the Knowledge Base. Now, when a KB is connected, a GROQ query for report documents gets a pointer to kb_knowledge_base_read instead.
The guard polices numbers, not words. The model can read any document over GROQ, including a withheld claim, and paraphrase it: "heavier than the incumbent" gets through, "4 GB" doesn't. I chose that on purpose, because hiding withheld claims from the model would also hide the context it needs to explain them. But it's a real limit, and the demo's answer shows it.
The measurement labels are ugly. The converter derives labels from the statement text (# MB free of 60167 MB). They're correct but hard to read. I left them alone because the validator matches on value, not labels.

5. Honest numbers, including where it's stricter than I'd like. 245 facts, 8 recommendations, 7 candidate tools, 3 runs, 12 tool areas and 4 reports. The structured validator shows 1 of the 8. The old text validator published 3. The two it now withholds are SigNoz and HyperDX, both REJECTs, and I checked why:

It's mostly the right call. The old validator only checked a number when a fact ID sat next to it. The new rule is that every number in a claim needs a structured source. SigNoz's "4 GB minimum RAM" and HyperDX's "4 GB + 2 cores" come from the vendors' docs, not from my estate. Web evidence has no document type yet, so those numbers have nothing to point at. The next step is a sourceClaim type (URL, fetched-at, quoted value) that numbersUsed can reference, the same way it references facts.
Some of it was my bug, and I fixed it. The first comparison also flagged 2026 and 14 from "Sep 14, 2026", 208 from a CISA bulletin ID (sb26-208) and a sandbox run ID read as a number. The identifier patterns now cover all of these, each with a test (27 tests in total).

Every withhold now names the exact number and says what's missing, instead of a regex failing to match somewhere in a sentence.

Sanity Project Details

Project ID: en9phc0q
Public dataset: toolsmith. Try it: https://en9phc0q.apicdn.sanity.io/v2025-02-19/data/query/toolsmith?query=*[_type=="recommendation"]{title,verdict}
Studio: https://toolsmith-kb.sanity.studio (members only; it exists so that Context will serve the dataset)
Knowledge Base: kbReXSyq7xIs ("Toolsmith"), built from *[_type == "report"] in the toolsmith dataset: 4 documents, 5 entries.
Context MCP endpoints: toolsmith (GROQ mode) and toolsmith-kb (Knowledge Base mode) in organisation omdh8fw7q. They need an org token with Context Viewer, so to try the agent without one, run --offline. That mode runs the same GROQ against the public dataset.

No Agent Session is attached: the build transcripts are full of my network's internal details. The code and the dataset are both public instead.

Ollama keep_alive -1 hangs every other model: a silent deadlock on a 6 GB GPU

Christian Anderson — Fri, 25 Sep 2026 08:00:14 +0000

My local LLM box is one machine with an RTX 2060. That's 6 GB of VRAM, and with the main model loaded there's about 1 GB left. Everything in the homelab that wants a local model talks to the one Ollama instance on it: the profiles of my agent platform, a user-modelling service, Home Assistant, a listing tool, and Open WebUI.

Twice now, every one of those clients has stopped getting answers. No crash, no error, nothing in the log. This is what was actually going on, including the fix I was sure about that turned out not to be one.

The first time: 26 August

Every agent turn hung. Not slow — hung. The requests for the main model sat open until each client gave up on its own schedule: 9 minutes on one, 3 minutes on another, 150 seconds on a third.

What I could see:

nvidia-smi read 181 MiB of 6,144 used, 0% utilisation. The GPU wasn't struggling. It was idle.
/api/ps listed exactly one model: a CPU-only build of qwen2.5:7b, with size_vram: 0 and an expires_at in the year 2318. That's what keep_alive: -1 looks like — "never unload".
The Ollama log had nothing. No error, no model-load line, no request line.

That last one cost me the most time. Ollama's HTTP request log line is written when a request completes. A request that's waiting to be scheduled hasn't completed, so it doesn't exist in the log. No log line means in flight, not never arrived. And /api/ps only lists loaded models, so the model everyone was waiting for wasn't anywhere either.

The pinned model belonged to the user-modelling service. I'd pointed it at a CPU-only build on purpose, so it would stop fighting the agents for the card. It stopped fighting for the card and started blocking the queue instead.

The un-stick was one request, no restart:

curl -s http://<ollama-host>:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{"model":"<pinned-model>","keep_alive":0}'

That returned "done_reason":"unload". The queued request for the main model then loaded in 52.8 seconds and answered, and the embedding model loaded straight after it. The whole queue had been stuck behind one pin.

The fix that wasn't

The obvious suspect was how many models Ollama will keep loaded at once. I set OLLAMA_MAX_LOADED_MODELS=2 in a systemd drop-in, restarted, and ran the failure case on purpose: pin the CPU model with keep_alive: -1, then immediately ask for the main one. Before the change that hung past nine minutes. After it, HTTP 200 in 13.4 seconds, with the pinned model evicted.

I wrote it up as fixed. It wasn't.

The second time: 10 September

I went to run one of the agent profiles and it produced nothing for about 15 minutes. Same signature: expires_at in 2318, size_vram: 0. The quickest test turned out to be asking for the loaded model and then for any other one:

the pinned model: 200 in 1.7 s
any other model: no response after 60 s

My first note said the setting must have reverted. I checked the host. It hadn't. OLLAMA_MAX_LOADED_MODELS=2 had been in place, and active, since 26 August. That setting was never the protection.

Worth knowing too: Ollama's default for that setting isn't 1. The documented default is three models per GPU. The number of slots was never the problem.

What's actually happening

It's VRAM. On a 6 GB card with about 1 GB free, a second model can't load until the first one leaves. Ollama's scheduler doesn't refuse the request when there's no room — it waits for room. A model pinned with keep_alive: -1 never leaves, so the wait never ends.

No environment variable fixes that. None of them can make VRAM, and none of them can override a keep_alive a client explicitly asked for.

The part that made it properly bad: a hang never triggers a fallback chain. My agent profiles each have fallback providers configured, and those fire on an error. A request that just never answers isn't an error. So every profile on a local model was dead, the fallbacks sat unused, and nothing anywhere reported a fault. I found it because I happened to try to use one.

It comes back within hours

I cleared the pin at 11:39. By 14:07 a different model, plain qwen2.5:7b, was pinned to 2318. Nothing in my agent profiles or scripts referenced the first one, and none of them set keep_alive at all. So this wasn't a leftover. Something is actively doing it.

I still don't know what. I didn't pin it, and Ollama doesn't log request bodies. The clients I could see connected were the listing tool, Home Assistant, the agent host, and Open WebUI running on the box itself. Open WebUI has a per-model "Keep Alive" setting and Home Assistant's Ollama integration exposes one too, and both accept -1. Those are my two suspects. Neither is proven.

The guard's own log narrows it a little. The expiry timestamps on the pins it clears overnight almost all fall at about 18 and 48 minutes past the hour, so whatever it is runs on a half-hourly schedule.

The actual fix: a guard, not a setting

A small Python script on a cron every 15 minutes. It reads /api/ps and unloads any model whose expiry is more than 48 hours away. A real keep_alive is minutes or hours (mine defaults to 30 minutes), so anything further out was pinned, not scheduled. The core of it:

for m in (_get("/api/ps") or {}).get("models") or []:
    exp = datetime.fromisoformat(str(m.get("expires_at")).replace("Z", "+00:00"))
    if (exp - now).total_seconds() / 3600 > MAX_SANE_HOURS:   # 48
        _unload(m["name"])   # POST /api/generate {"model": ..., "keep_alive": 0}

Two decisions in there that matter:

It only touches pins. A model with a normal expiry is left alone even if it's hogging the whole card, because that one will leave on its own.
If Ollama is unreachable it exits cleanly and says so. A down server is a different fault with its own monitor, and this guard shouldn't page for it.

It has a --dry-run flag, and it's deterministic. No model is involved in deciding what to unload.

It's still firing. On the morning I'm writing this it unloaded qwen2.5:7b, "pinned for 292 years", at 09:45, 12:15 and 12:45. The log shows 408 unloads since it went in on 10 September, between 20 and 43 a day. Every one of those was a window where any other model would have hung.

What I'd tell anyone running Ollama on a small card

A hang with no log line isn't a quiet server. Ollama logs a request when it finishes, so a missing line means the request is still waiting.
/api/ps won't show you the model everyone's waiting for. It lists loaded models only. Look for expires_at years in the future and size_vram: 0 on the one that is loaded.
An idle GPU during a hang means a scheduling problem, not a resource problem. 181 MiB used at 0% utilisation isn't a card under strain.
Fallback chains fire on errors, not on silence. If your failover depends on the primary failing loudly, a hang gets past it. Put a timeout on the client, or watch for this directly.
Test the fix against the real thing, then check again later. My regression test passed on the night, and the problem came back two weeks later with the setting still there.
If one client can pin the card, assume one will. On a shared single-GPU box, clearing the pin on a schedule is more reliable than hunting down every client that might set it.

The un-stick is one curl. Knowing that you need it is the hard part, because nothing tells you.

Cryptonym Desk on Sanity: moving a word list into a dataset found two bugs

Christian Anderson — Thu, 24 Sep 2026 18:35:39 +0000

This is a submission for the Sanity Challenge, Path Two: Vibe-Code Something Strange

What I Built

Cryptonym Desk gives you a spy file for your agent. You type in films, anime and characters you like. It gives you back three names: a CIA-style cryptonym (a real two-letter office digraph plus a word that means nothing, like AEDINOSAUR or LIENVOY), a working alias made by grafting your names together at their vowels, and an ADJECTIVE NOUN field codename.

The first version was one 29 KB HTML file with every word list hard-coded in a <script> tag. For this challenge I moved the lists into a public Sanity dataset and rebuilt the page in Astro, which reads that dataset when the site is built.

It's for people who name things: agents, bots, side projects, D&D characters. It doesn't take itself seriously. The one thing I kept strict is that nothing you type ever leaves the page.

Demo

Live: https://casareanderson.github.io/cryptonym-desk-sanity/
The corpus, rendered from Sanity: https://casareanderson.github.io/cryptonym-desk-sanity/corpus/

Code

https://github.com/casareanderson/cryptonym-desk-sanity (MIT). The original single-file version is at https://github.com/casareanderson/cryptonym-desk.

My Build Process

The tool: Claude Code, in a terminal, not an IDE. I gave it a short brief and it did the build end to end: import, schema, Astro port, tests, GitHub Action, deploy. I reviewed the result. What went wrong along the way is more useful than what went right, so this section is mostly that.

1. The schema came first, and the data got cleaner as a result. The three document types are digraph (code, provenance note, retired flag), wordBank (bank, value, weight) and preset (label, seed list). The corpus came to 103 documents. Just turning the arrays into documents exposed a bug: MERIDIAN was in the noun list twice, so it had been drawn twice as often as any other noun. In a JavaScript array you never notice that. As separate documents, it stands out immediately.

2. A test written from the README failed, and the README was the thing that was wrong. The README says the splitter cuts Spiegel into Spie·gel and Kusanagi into Ku·sa·na·gi, so Spiegel × Kusanagi gives Spienagi. The agent wrote a test for exactly that. It failed. The regex had always kept one consonant after each vowel group, giving Spieg·el and Kus·an·ag·i, so across 200 seeds that pair only ever produced Spiegagi. The README's own example could not happen. The obvious fix was to change the test to match the code. Instead we treated the documented behaviour as the design and the regex as the bug. The code now matches the README, and there's a test that the README's example can actually be produced. The cost: the same inputs now give different aliases than the original desk did.

3. The weight field was decoration at first. The schema had it and the generator did weighted picks, but every entry weighed 1, so none of it did anything. I only noticed when the corpus page showed a column of ×1. The fix was an edit in the dataset, not the code: plain SECRET weighs 5 and CODE WORD weighs 0.5. That edit is also the end-to-end proof. The corpus revision printed on the live site went from 211eb8a to 4ca7fee on the next build, with no code change.

4. Every record now carries the corpus revision. Records have always been seeded (same inputs plus the same salt gives the same record), which is what makes a "file reference" mean something. Once the words live in a dataset, the same inputs can produce a different record after someone edits a bank. So the page hashes every document's _rev into a 7-character revision and prints it on the record and in "Copy record". Without it, "same inputs, same record" would quietly stop being true.

5. The build refuses a corpus that would break the page. If a bank is empty, a bank name is unknown, a weight is zero or no digraphs are active, astro build fails, so the page can't ship and then throw on click. The Studio schema enforces the same rules earlier (positive weights, uppercase 2–3 letter codes, at least two seeds per preset).

6. What I deliberately didn't do. The corpus is fetched at build time only. The page makes no runtime calls to Sanity, because it has always promised that nothing you type leaves the browser, and I'd rather keep that promise than add a live query. Edits reach the site through a GitHub Action that runs on push, on demand and nightly. A Sanity webhook would be faster, but it would mean storing a GitHub token inside Sanity, and a nightly rebuild is plenty for a word list.

7. Where it stopped, and the update. The first version was a read-only frontend, and I said the obvious next step was a Workflow that lets people propose words, with a person approving each one into a bank. That's now built (update, 25 September):

Update: word proposals, a workflow kept as data

Every proposal is a wordProposal document in the same public dataset: the word, the target bank, a weight, a rationale, and a status that moves through proposed → in review → approved → merged (or rejected, which can be reopened). Each move appends to a history[] {status, at, by, note} array on the document, so one GROQ query answers "who approved CISTERN, and when?". There's no second database for the workflow state.

One rules module, three surfaces. An App SDK review board, Studio document actions and a CLI all call the same dependency-free src/lib/proposals.js. None of them can skip review: proposed → merged is refused. A rejection needs a reviewer note.
The board is deployed as a Sanity App SDK app ("Word proposals"), and organisation members can open it from the Sanity Dashboard.
The board updates live. It uses useQuery from @sanity/sdk-react, which subscribes to changes. A move made from the CLI showed up on the open board in about 2.6 seconds, with no reload.
The merge is deterministic code in one transaction. It does a createIfNotExists for the new wordBank entry (a predictable _id, so a double click lands on the same document) plus a patch to the proposal pinned with ifRevisionID. If two reviewers merge at once, only one succeeds. I tested this against the live API: the stale merge got a 409 and wrote nothing.
It catches the bug the port found. A proposal for a word already in the bank links to the existing entry instead of creating a second one. That's exactly how MERIDIAN ended up drawn twice as often in the original.
Proposals don't change the corpus revision. Moving cards around leaves everyone's CORPUS stamp alone. Only a merge changes the words, so only a merge changes the stamp.

The loop, run for real: CISTERN went proposed → in review → approved → merged, and it's now in the noun bank of the live site. There are 15 nouns instead of 14, and the corpus revision changed with no code change.

What went wrong: sanity@latest (v6) needs Node 22.12 or later, and the box has Node 20. The v4 CLI's unattended init --template app-quickstart asks for flags it then refuses to accept together, so the app was written by hand from the template the CLI bundles. A project robot token can deploy a Studio but not an App SDK app, which needs an organisation-level grant. And the tradeoff of a public dataset: proposals, rationales and reviewer notes are world-readable, like the rest of the corpus. The Studio actions patch the published document and are disabled while a draft exists, so an unsaved edit can't be silently left behind.

Tests: 19 in total (10 new). They cover the transition table, skipped-review refusal, the rejection note, the predictable merge _id, the duplicate guard and the one-transaction revision pin.

One honest gap in the data is still there: each digraph's provenance note is a generic "real digraph from declassified material" line, not an individual citation.

The rest of this section is from the original build.

Small things that went wrong: The Studio's first sanity deploy stopped and asked for the new app ID to be pinned in sanity.cli.js. The headless screenshot step first tried a Python Playwright that wasn't installed, then used the Node one. And GitHub Actions now warns that Node 20 actions are deprecated.

Tests: 9 node:test cases. They cover determinism, the revision on each record, weighted-pick distribution (10,000 draws), split parts rejoining to the original word, the README example actually appearing, and the corpus validator rejecting each kind of bad data.

Sanity Project Details

Project ID: en9phc0q, dataset production (public read)
Read the whole corpus, no token needed: https://en9phc0q.api.sanity.io/v2024-01-01/data/query/production?query=*%5B_type%20in%20%5B%22digraph%22%2C%22wordBank%22%2C%22preset%22%5D%5D
Schema: studio/schemaTypes/index.js
The GROQ query the page is built from is printed at the bottom of the corpus page.

Agent Session

Not embedded. The session that built this ran on my homelab box, and the transcript is full of private network detail that has nothing to do with the build. I'd rather leave it out than publish a heavily redacted version. The repo's commit history, tests and this write-up are the record.

Piper TTS over Wyoming: the frame format and empty model that broke my podcast

Christian Anderson — Thu, 24 Sep 2026 12:55:31 +0000

I wanted my own articles as audio: something I could listen to on a walk, as a conversation between two voices rather than one voice reading. Not a cloud service. A thing I run, on the hardware I already have, with no GPU.

It works now. The first episode came out at 6 minutes 46 seconds, 58 turns, normalised to -16 LUFS. Getting there took two bugs that both looked like something else, and a licence check that ruled out the voices I wanted most. Here is what actually happened.

The shape of it

Four stages: fetch → script → voice → mix. Each one writes its output to disk (the article JSON, the dialogue turns, one WAV per turn, the final MP3), so any stage can be re-run on its own. That turned out to matter more than anything else in the design, because every bug below was found by re-running one stage, not the whole thing.

Script: a local 7B model writes the dialogue, one article section at a time.
Voice: Piper, spoken to over the Wyoming protocol, with two stock British voices, one per speaker.
Mix: ffmpeg concat plus short pauses, then loudness normalisation.

Voice: Piper over Wyoming, no new service

I already had a wyoming-piper service running for my home voice assistant. It turns out that if you send it a synthesize request naming a voice it has never seen, it downloads that voice on first use. So two distinct speakers cost nothing: no new container, no GPU, no API key.

Measured on a 2-core container: 8.4 seconds of audio in 2.9 seconds of wall time, a real-time factor of about 0.35. A 15-minute episode is roughly five minutes of synthesis. Clips are cached by a hash of (voice, text), so fixing one line re-renders one line.

Bug one: the frame has three parts, not two

The first version of my Wyoming client read a JSON header line, then read the audio payload after it. It failed straight away with:

Extra data: line 1 column 62

That reads like a broken server sending malformed JSON. It wasn't. A Wyoming event can carry three parts: the header line, then an optional block of JSON data, then an optional binary payload. The header announces the sizes of both. If it declares a data_length, that data is its own block of bytes after the newline, not a field inline in the header. My reader skipped it, landed in the middle of a JSON object, and choked.

The fix is to read what the header tells you to read, in order:

def _read_event(f):
    line = f.readline()
    if not line:
        return None, b""
    head = json.loads(line)
    dn = head.get("data_length") or 0
    if dn:
        head["data"] = json.loads(f.read(dn))
    pn = head.get("payload_length") or 0
    return head, (f.read(pn) if pn else b"")

The audio-start event puts the sample rate, width and channel count in that data block, which is why skipping it was fatal rather than cosmetic.

Bug two: the model that answered in a `thinking` key

The first full run produced a cheerful log line: intro: 0 turns. No error. The natural assumption is a bad prompt, and I spent time there first.

The real cause was the model. My default local model is a thinking model, and I was using a variant whose Modelfile is meant to switch thinking off. Measured through Ollama's API: message.content was an empty string, and the entire 200-token budget had gone into a separate thinking key. The "no thinking" Modelfile parameter was accepted and ignored.

Two fixes. Send "think": false in the request body (the API-level switch does work where the Modelfile one didn't), and refuse to return an empty answer silently:

content = msg.get("content") or ""
if not content.strip() and (msg.get("thinking") or msg.get("reasoning")):
    raise RuntimeError(f"{model!r} spent its whole budget in a `thinking` "
                       "block and returned no content.")

An empty content from a thinking model looks identical to a model with nothing to say, and everything downstream just counts zero turns. That sends you to the prompt when the problem is the model. Now it fails loudly and says why.

I also switched models on measurement rather than habit. On my 6 GB card, which is already mostly holding an image model, the default ran at 5.3 tokens/s (about 90% on CPU). A plain 7B non-thinking model ran at 12.6 tokens/s. Faster, and it actually returns content.

The small 7B problems

A 7B model writing structured dialogue needs guard rails:

Ollama's JSON mode returns an object, and I wanted an array, so the model wrapped the array in a key it invented, differently each time. Asking plainly and parsing defensively was more predictable.
Sometimes it wraps each object in quotes, which is invalid JSON. A last-resort regex pulls out the speaker/text pairs.
A check() step runs before synthesis and rejects monologue turns, empty turns, unknown speakers, URLs and bullet points. That's cheaper than finding them after five minutes of CPU spent speaking them.

The rule that keeps it honest: the source sets the ceiling

The model writes dialogue; it never supplies facts. Each prompt carries only the section it's allowed to talk about. And the target length is capped at 1.15 × the article's word count. An 828-word article becomes about six minutes, not fifteen. The only way to stretch a short article into a long episode is to make things up, so the pipeline refuses to ask for it, and says so.

The mix

ffmpeg -f concat copies the PCM straight through, so stitching takes seconds, not minutes. I insert a 320 ms pause between lines from the same speaker and 520 ms on a handover, which is enough to hear the conversation change hands. Then loudnorm=I=-16, because Piper's two voices aren't equally loud out of the box.

The licence check

I wanted to go further and have one of the voices be mine, cloned from an old recording. The two most-recommended cloning models were out as soon as I read the licences: XTTS-v2 is under a non-commercial licence (CPML), and the F5-TTS weights are CC-BY-NC. This podcast links to things I sell, so neither was an option.

I used OpenVoice v2 with MeloTTS instead (both MIT). The lesson there was a different one: OpenVoice changes timbre, not rhythm. My vocal colour on top of Piper's cadence still sounded like a machine. No better sample fixes that; the cadence comes from the underlying TTS. Kokoro (Apache-2.0) is the next candidate. Measured on CPU, it speaks the same line about 24% slower because it takes the pauses a person would.

What I'd tell you if you're building one

Read the protocol spec for framing before writing a client. A three-part frame read as two gives you an error that blames the server.
Treat empty model output as an error, not an answer. Check for a thinking or reasoning field when content is empty.
Write every stage to disk. Being able to re-run one stage is what turned each of these bugs into a short investigation.
Read the model licence before you fall in love with the demo.

Authelia passkeys with TOTP as break-glass: why the config diff was empty

Christian Anderson — Thu, 24 Sep 2026 08:00:03 +0000

Everything behind my front door goes through one login portal: photos, password vault, the VPN control plane, my agent dashboard. It runs Authelia, and for a long time the second factor was a TOTP code out of an authenticator app.

Now it's a passkey, with TOTP kept only for the day I lose the authenticator.

I went looking for the configuration change that expresses that. There isn't one. The diff is empty, and understanding why it's empty is the whole point of this post.

What the policy actually says

The relevant lines, unchanged before and after:

access_control:
  default_policy: 'two_factor'
  rules:
    - domain: '*.example.internal'
      policy: 'two_factor'

And every OIDC client registered with it:

      - client_id: 'some-app'
        authorization_policy: 'two_factor'

two_factor is a count, not a method. It says: a password, and then one more thing you have proven you control. It does not say which thing. Authelia will accept any second factor the user has enrolled, and the user decides which one to present at the prompt.

So "passkey normally, TOTP only if I'm locked out" is expressed by what I enrolled, not by anything in the file. My config has no webauthn: section and no totp: section at all — both run on v4.39 defaults, because nothing needed saying.

If you came here expecting a config diff, that expectation is the misunderstanding. It was mine too.

Why the passkey is the normal one now

TOTP is a shared secret. When you scan that QR code, the server keeps a copy of the seed — something that can regenerate your codes. And the code itself is six digits that can be read over your shoulder, screenshotted, typed into the wrong box, or relayed by a real-time proxy: a fake login page that takes your code and replays it to the real portal inside the 30-second window. You do everything right and still lose.

A passkey is different in kind, not in degree:

The private half never leaves the authenticator. The server stores a public key, so a database leak gives an attacker nothing to replay.
The browser binds the credential to the origin. A lookalike domain cannot invoke the passkey for the real one — not "shouldn't", cannot. The relay attack above has nothing to relay.

That second property is the one that matters. Most 2FA advice is about making the user more careful. This one removes the situation where carefulness was the defence.

Why TOTP is still enrolled

Because recovery beats purity.

Every service in the house sits behind this portal. If the passkey lives on one device and that device goes in a canal, an untested recovery path is not a recovery path. TOTP as break-glass is the honest trade: normal logins get the unphishable factor, and there is a second door I have actually opened.

The uncomfortable part, stated plainly: a break-glass path is a path. An attacker who can steer me to the "use a code instead" link gets phishability back. Keeping TOTP means keeping that floor. I accept it, knowing it's a real cost and not a technicality — the alternative is a single device standing between me and everything.

Where 2FA can't go

The other thing reading that config taught me is that two rules in it are deliberate holes, and both are the same shape.

A vault whose native clients aren't browsers. The browser extension and the mobile apps talk to API paths directly. They can't follow an HTML redirect to a login portal, so gating those paths doesn't secure them — it breaks every client. The rule gates the browser-only admin panel and bypasses the rest:

    - domain: 'vault.example.internal'
      resources: ['^/admin.*$']
      policy: 'two_factor'
    - domain: 'vault.example.internal'
      policy: 'bypass'

Webhook endpoints. A webhook is a machine posting to a URL. There is no human to prompt.

The generalisation: an SSO portal authenticates browsers. Anything that isn't a browser needs its own mechanism — a token, an API key, a signed payload — and pretending otherwise produces either a broken app or a bypass rule you wrote in a hurry and never revisited. Write the rule narrow, scope it to paths rather than whole hosts, and put the reason in a comment next to it. Mine has one, and it's the only reason I trusted it a year later.

The takeaway

Two factors is a policy. Which two is an enrolment. Those live in different places, and only one of them is in your config file.

If your portal already says two_factor, you are one enrolment screen away from unphishable logins — no redeploy, no YAML, and no reason to put it off as a project.

eBay sold prices without the paid API: I built my own sold-comps engine and measured it

Christian Anderson — Wed, 23 Sep 2026 08:00:03 +0000

The paid meter was running out

I had a paid eBay sold-comparables service. 250 requests a month. Some products
need more than two hundred listings before you get a usable price cluster, so
one wrong query burns a week's ration. I started wondering: do I actually still
need it, or could I grow my own?

That question is empirical. The repo that came out of it is
sold-shadow.

The premise (and the thing that made it possible)

eBay's completed-listing search has been dead to automation for years. Free
alternatives show asking prices, which are opinions. Paid services resell
closed-access sold data, and they're metered.

But an ended listing does not disappear from the Browse API. A GET /item/{itemId} on a listing that finished last week returns HTTP 200 with the
itemEndDate field set, and the payload tells you whether anything actually
sold. The signal is straightforward:

OUT_OF_STOCK, estimatedRemainingQuantity 0, estimatedSoldQuantity >= 1  -> SOLD
IN_STOCK,     estimatedRemainingQuantity 1, estimatedSoldQuantity 0     -> ended unsold

Tested on thirty real listings re-fetched 6-8 days after first capture: 30/30
returned 200, two had ended, one of those had sold. That proved the data doesn't
evaporate.

This uses a client-credentials token — the Browse API application-level flow.
No user consent, no scraping, nothing that touches the API License Agreement
edge. The same credentials that run publishing and orders.

How it works

You tell it what to track. Not categories, not everything — the API quota is
finite, and a corpus of products you never sell is worth zero. Each track
command stores one search phrase.

A daily sweep re-fetches every open listing, checks whether it ended, and
records the outcome if it did. Self-capped at 600 calls per day against a 5,000
Browse allowance. When a sale arrives, the listing becomes a permanent record:
item ID, title, sold price or best-offer ask, the query that found it, and a
flag for whether the listing accepted offers.

You can also feed it your paid service's API key and run compare, which
fetches the same queries from the paid source. It caches responses for 24 hours
so only the first run each day costs real quota, and it shows you exactly how
far your own corpus is from matching the one you pay for.

What the numbers said after a week

After about seven days of daily sweeps on one install (measured evening of
2026-09-12):

3,190 listings tracked across 54 queries
14 confirmed sold
9 of those were best-offer listings (upper bounds, not real prices)
5 genuinely usable sold comparables
5 queries out of 54 had at least one usable comp — 9% coverage

The switch-off bar I wrote into shadow.py is 80% coverage within 15% of the
paid source's median. The honest verdict: not close. One query had a real
cluster (four Casio FX-CG50 sales in four days). The rest were n=1.

That is not a soft disclaimer. A corpus is either mature or it is nothing, and
you cannot know which it is without measuring it against truth. The compare
harness was built for that purpose, and putting it in the same repo forces the
honest conversation: can I stop paying yet? Most of the time, right now, no.

The things that bite (four of them)

No backfill, ever. The search endpoint returns active listings only. Any
sale that completed before your first sweep is invisible. The highest-value
action you can take with this repo is start it today and walk away for a month.
Time is the only thing that grows a corpus.

Best-offer sales are ceilings, not prices. eBay never exposes the accepted
offer on a best-offer listing. The ended item still shows the ask. On the data
above, 9 of 14 sales were flagged as best-offer listings — excluding them loses
64% of your comps, including them prices against a number nobody paid. They are
stored with a flag, excluded from sold_comps() by default, and returned
separately by best_offer_comps() so the caller can decide.

The flag itself is permissive: it is set when BEST_OFFER appears in
buyingOptions, which means the listing accepted offers, not that this
particular sale went through one. On eBay UK most used fixed-price listings are
best-offer. One product (the same Casio FX-CG50) had a clean sale at £89.99 and
three best-offer asks at £80.00, £83.90 and £88.99 — the discarded asks sat
beneath the clean sale. Excluding them was not the conservative choice it
looked like. compare therefore counts them in a separate bound column.

Auctions lie about their end date. itemEndDate is set at listing creation
for auctions, not when they finish. The assumption that "live listings have a
null end date" is true for fixed-price only. An earlier version of this code
marked every auction as ended-unsold while it was still taking bids, and never
revisited the row. Auctions could never contribute a comp. Since auctions are
the cheap end of a used market, the corpus was biased upward by construction.
The fix: a future end date now leaves the row open.

One search string finds one slice. eBay UK holds roughly twenty active
listings of any mid-volume used product at once, and no single query sees all
of them. The same case is listed as "Argon ONE M.2", "Raspberry Pi Argon One
With" and "Argon One V2 aluminium". On the same install, adding alternative
phrasings for the same product changed the count:

Apple Magic Mouse: 210 → 388
Casio FX-CG50: 18 → 104
Xiaomi TV Box S: 36 → 81
Raspberry Pi Argon ONE: 23 → 23 (no more to find — phrasings only help if the volume exists)

That was +640 listings for +100 API calls in a single sweep. Two rules from that
run: keep a phrasing to eight words or fewer (a long string ANDs every word and
returns less), and cap it at three per product so a sweep of thirty products
stays under a hundred searches.

What the GitHub repo gives you

Python 3.11+, stdlib only — no dependencies. Three commands: track to
enrol a search phrase, sweep as a daily cron job, comps to see what has
matured. A compare harness that talks to a paid service's API so you can
measure the gap. An MIT licence.

The repo is
https://github.com/casareanderson/sold-shadow.

The takeaway

This is not a story about replacing a paid tool. It is a story about building a
thing, being forced to measure it honestly, and discovering that the hard
constraint is not the API quota — it is calendar time. You cannot backfill a
used-goods market from the Browse API. You can only watch and wait.

The sold-shadow code works. The corpus, after a week, does not. Both of those
statements are visible in the same output, and that is the point.

Start the sweeps today. Check back in a month.

Naming AI agents: what the CIA's declassified cryptonym convention taught me

Christian Anderson — Tue, 22 Sep 2026 08:00:04 +0000

I have a lot of agents running on my homelab, and they were all called things like
writer, coder, sec. Fine. Boring, but fine.

Then I went looking at how a real service names things, and found out that one flat
name is not how it's done at all. You get several, and they do different jobs. That
turned into a small single-file web toy — Cryptonym Desk
— and the interesting parts were not the ones I expected.

Let me start with the honest bit, because it shapes everything after it.

This is not a market gap

Name generators are one of the most crowded categories on the web. I knew that
before I wrote a line. I have ten products on a store with effectively no sales,
and the lesson I keep re-learning is that the bottleneck is distribution, not
ideas.

So this was built on a stated basis: a cheap, shareable link. Not revenue. Not a
product. Something with one good detail in it that someone might pass on. If you
are building your fifteenth side project this year, it is worth writing that
sentence down before you start, because it changes what you optimise for.

What follows is what actually turned out to be interesting.

Three names, because one name is wrong

The convention I borrowed from is the cryptonym system in declassified US
intelligence files. The shape is:

A two-letter digraph, then a word chosen to mean nothing.

AEDINOSAUR. QKACTIVE. LIENVOY.

The digraph is the part people miss. It isn't decoration — it encodes which
office or country the material belongs to, and the word after it is deliberately
meaningless, because a meaningful codename leaks. If you call an operation
NIGHTFALL you've already told me something about the mood of it. AEDINOSAUR
tells you nothing, which is the whole point.

The digraphs on the page are real ones pulled from declassified material, which is
the detail that makes it feel grounded rather than generated.

Then two more names, because the cryptonym can't do every job:

A working alias — a human-looking name, for when the agent has to appear as a person in a log or a commit.
A field codename — ADJECTIVE NOUN, for traffic that must not carry the alias at all.

Three names, three purposes. That structure is what makes it not-another-generator,
and it came entirely from reading how the real thing works instead of guessing.

The alias is a mashup, and the seams are ugly on purpose

This is the part I'd want to read about, so here it is properly.

The working alias is not picked from a list. It is a graft: two source names are
split at their vowel groups and the front of one is joined to the tail of the other.

Spiegel  +  Kusanagi   ->   Spienagi

The card shows you which two names were cut, because the join is most of the fun.

Now the bit that surprised me. My splitter is crude. It finds vowel groups with a
blunt regex and cuts there, with no understanding of onsets, codas, or where a
syllable legally breaks in English. My first instinct was to fix that — pull in a
proper syllabifier, get the linguistics right.

I tried it. The names got worse.

A correct syllabifier gives you tidy, well-formed seams, and a tidy seam produces a
name that sounds like it was always going to exist: plausible, forgettable, dull. The
crude cut sometimes lands mid-cluster and produces something with a slight wrongness
to it — and that is what makes a name sound like a codename rather than a
colleague. The bad algorithm is the feature.

The only correction I kept is collapsing a tripled letter at the join, so you get
Rippley and not Ripppley. That one is just a typo, not a texture.

There's a general lesson hiding in there: when you're generating something whose
purpose is to feel a certain way, "more correct" and "better" can point in opposite
directions. Measure the output, not the algorithm's respectability.

The records are seeded, and everyone finds this weird

Type the same inputs in twice and you get the same record back. Same cryptonym,
same alias, same codename.

That's deliberate — the inputs plus a salt seed the generator, so a given set of
material always rebuilds the same identity. It means a record is reproducible: you
can hand someone the inputs instead of the output. Variety comes from an explicit
"Issue new record" action, not from mashing the same button and hoping.

Every single person who has used it has hit this and assumed it was broken. I put it
in the README, and I'd now argue that if a design decision reliably surprises users,
the README is not where the explanation belongs — the UI should say it. That's the
next change.

The deployment trap: an artifact is not a web page

The thing also exists as a Claude artifact, and I assumed the artifact HTML and the
GitHub Pages HTML would be the same file. They are not, and the difference cost me
a confusing ten minutes.

The artifact host supplies things I'd taken for granted:

the doctype
the charset and viewport meta tags
a small CSS reset

Copy that file to a static host and it renders subtly wrong — wrong scaling on
mobile, a mystery margin round the edge. The standalone version needed all of the
above added back plus an explicit body { margin: 0 }.

So if you publish a page in both places, don't copy one to the other. Keep the
standalone file as the real one and re-wrap for the artifact, not the reverse.

What it's made of

One index.html. No dependencies, no build step, no framework, no network calls —
nothing you type leaves the browser. Three fonts doing three jobs: Antonio for
display, Share Tech Mono for file numbers, and Public Sans for body text, which is
an actual government typeface and therefore exactly right for the subject.

It's MIT, it's about a screen and a half of JavaScript, and you can read all of it
in one sitting: github.com/casareanderson/cryptonym-desk

Go and issue yourself a record. If the alias it grafts for you is slightly wrong in
a way you like, that's the crude splitter doing its job.

Spotify + djay on an iPad as a hands-off party DJ: my child's baptism, and an agent that ignored the brief

Christian Anderson — Mon, 21 Sep 2026 08:00:03 +0000

My child's baptism party was at a pub. I wanted proper music — music jukebox, not a Spotify "Chill Hits" playlist — and I did not want to spend the day standing over an iPad.

This is how that worked: Hermes (the agent that runs on my homelab) and Claude Code built the playlist between them, and the iPad just played it. It also covers the part that didn't work, because the first version the agent built was wrong in a way that would have been embarrassing at a christening.

The brief

I wrote a structure, not a list of songs:

warm R&B while people arrive (Jorja Smith, Tems)
afrobeats as the energy lifts
UK garage for the peak
afro / reggae
an outro to close the day

And a reference: Chuckie Online's mixes — his apartment life mix and his BBC 1Xtra R&B Slow Jamz mix. The instruction was that his selection is the base, not a flavour.

Attempt one: the agent ran the wrong job

Hermes has a Spotify skill I'd built for weekly mixes. Given the baptism brief, it didn't follow it. It ran the weekly-mix defaults instead and produced 60 tracks of 90s and modern rap, with duplicates and explicit tracks. At a baptism.

The cause was a gap, not a bad model: the Spotify tool had a "make a playlist from these genres" mode and no "build from a brief or a reference tracklist" mode. With no tool that matched the job, the agent reached for the nearest one and reported success.

Two more things the brief itself got wrong, because an LLM had helped write it:

Credits were wrong. "Essence" is Wizkid featuring Tems; "Location" is Dave featuring Burna Boy; "Re-Rewind" is Artful Dodger featuring Craig David. Every credit had to be checked by search.
The artist links were invalid IDs.

Attempt two: rebuilt from the brief, by hand

I took it off Hermes and rebuilt it with Claude Code, using Hermes's Spotify credentials and box:

Reference tracklists. A SoundCloud mix page has no tracklist. The YouTube upload of the same mix does, in its chapters. The BBC programme pages list every track for his 1Xtra episodes.
Baptism rules. Drop anything that only exists as an explicit version, and drop the bedroom slow jams. There are certain 90s R&B songs your nan shouldn't hear at a christening.
Blocks, not a shuffle. The final running order:

Block	Opens with	Length
Warm R&B	Essence — Wizkid, Tems	40 min
Reggae / lovers rock	I Feel Good — Beres Hammond	56 min
Afrobeats	Last Last — Burna Boy	56 min
Afro house	I Adore You — HUGEL, Topic	43 min
UK garage	Re-Rewind — Artful Dodger	26 min
Old-school dancehall	Heads High — Mr. Vegas	42 min
Modern R&B	Is This Real Love? — Debbie	32 min
Slow jams	When I See U — Fantasia	61 min
Outro	Where Did I Go? — Jorja Smith → By Your Side — Sade	18 min

101 tracks, 6.2 hours. It peaked at 233 tracks and 14.5 hours before I cut it back to the order I actually wanted.

The Spotify API fights back

It's worth knowing about these if you're building anything on Spotify's API in 2026:

A removed endpoint returns 403, not 404. Spotify renamed several endpoints in February 2026 (/playlists/{id}/tracks → /items, POST /users/{id}/playlists → POST /me/playlists). The old paths answer 403 Forbidden, which looks exactly like a permissions problem. I lost hours re-consenting scopes before checking the path.
Search caps at 10 results for a development-mode app, and year: / genre: filters return nothing. Era and style have to come from how you phrase the search.
You can't read other people's playlists' tracks in dev mode, so "copy Chuckie's playlist" isn't possible. The YouTube chapters were the workaround.
Rate limit. About 250 searches in one evening got a 429 with a retry-after of 77,746 seconds — 21.6 hours. A second Spotify app didn't help, because the throttle is per user account, not per app. So the rest was scheduled on Hermes's box: one timer to add about 40 Chuckie tracks the next evening, and a second to reorder the playlist into the final blocks, which only ran once the first had succeeded. Both ran unattended the night before the party.

The playback: djay on an iPad, hands off

The player was djay Pro on an iPad, logged into Spotify inside djay, on the 7-day free trial. So the playlist Hermes's box built was just there in djay's library, and djay's Automix did the transitions. The iPad connected to the pub's speaker over Bluetooth, and it held up with no drop-outs.

You can also wire it: a USB-C (or Lightning) to 3.5mm/aux adapter into the pub's mixer or amp. Wired is the safer choice if the speaker is a long way away or the room is full of phones. It also has no Bluetooth delay, and with a USB audio interface you get a headphone output to hear the next track before the room does — which is what you'd need to mix by hand.

The settings that made it hands-off:

Shuffle off. The order is the whole point.
Automix on, with short transitions (about 10 seconds). Long blends make two rappers talk over each other.
Max song length about 2:30–2:45. Most 90s tracks carry a minute of outro nobody dances to. That turns the 6.2 hours into about 4.5.
Auto gain and keylock on. 90s masters are much quieter than modern ones, and keylock stops voices going chipmunk when tempos are matched.
Explicit content off, repeat on, iPad auto-lock off, Do Not Disturb on. Explicit off is the belt-and-braces for a christening. Repeat means the music never stops. Auto-lock and Do Not Disturb are the two ways an iPad can kill the music mid-party.
On Bluetooth, don't mix by hand. You can't hear the next track before the room does, and the audio lags the screen. Let Automix do it.

The only thing a human does is steer: if the room is ready early, jump to the first song of the Afrobeats block; when people start leaving, jump to the slow jams, then the outro. The blocks mean that one tap changes the mood without breaking the flow.

What Hermes actually did, and didn't do

Did: hold the Spotify skill, credentials and scheduling that everything else ran on. The two timers that finished the playlist ran on its box, unattended, the night before the party.

Didn't: build this playlist. Given the brief, it ran the wrong job and called it done; the actual build was done by hand with Claude Code. The fix wasn't a better prompt. It was noticing that the tool had no mode for the job, so the agent improvised.

That's the lesson I keep relearning with agents: when there's no tool that fits the job, they don't stop and say so — they use the nearest tool and report success. Check the output against the brief, not against the agent's summary.

AI blog writing pipeline without made-up facts: how I post daily from real work

Christian Anderson — Fri, 18 Sep 2026 08:00:02 +0000

I publish a homelab post most days. I don't have a content team, I don't have a
backlog of pre-written drafts, and — this is the part that matters — I don't have
an AI cheerfully inventing things I "did" so it can hit a daily quota.

What I have is a small pipeline that turns work I actually did into posts, with a
strict rule about what the AI is and isn't allowed to do. This post is that
pipeline, explained — and, fittingly, it came through the pipeline itself. Let me
show you the machine, then prove it by telling you how this got made.

The problem with "just get AI to write your blog"

The obvious version of this is a disaster, and it's worth being clear about why.

If you point a language model at "write me a post about my homelab this week," it
will write you a post. It will be fluent, confident, and it will contain things
that never happened — a metric it made up, a feature you didn't ship, a lesson
from a bug you never hit. Not because it's malicious, but because it was asked to
produce content and it has no way to tell the difference between a fact and a
plausible sentence. A model told to fill a daily slot will fill it, truthfully or
not, and you won't be able to see the seam.

So the entire design of my pipeline is built to make that specific failure
impossible. One rule underneath everything:

The clever, non-deterministic part is allowed to write. It is never allowed to
decide what's true. Here's how that splits into stages.

Stage 1 — a deterministic radar finds real topics

Nothing about topic selection touches an AI. A plain script scans the actual
evidence of what I've been doing — commit history across my repos, what changed,
what shipped — and turns it into a list of candidate topics, each with the real
facts attached: the actual commit messages, the actual repo, the actual dates.

This is the crucial move. The list of things worth writing about is derived from
things that demonstrably happened, mechanically, with no model in the loop. If I
did nothing worth writing about this week, the radar says so — and the honest
consequence is fewer posts, not an invented one. A quiet week is allowed to be
quiet.

Stage 2 — the model drafts, from those facts only

Now the AI comes in, and its job is deliberately narrow: take one candidate
topic and its gathered facts, and write it up in my voice. It's handed the real
commit history and told, in effect, "only use this; if a detail isn't here, stay
general rather than inventing a specific."

It's a writer working from a brief, not a journalist making up a story. It phrases,
structures, and finds the through-line — the things models are genuinely good at —
but the raw material is all real, because it was gathered by Stage 1 and handed
over. The model never went looking for a fact, so it never had the chance to
fabricate one.

Stage 3 — two gates before anything is public

A draft is not a post. Between "the model wrote something" and "the world sees it"
there are two hard gates:

An automated leak gate. The draft is scanned for anything that should never be public — real IP addresses, hostnames, secrets, personal details. Anything that trips it gets quarantined, not published. This is a machine check, not a judgment call.
A human gate. I read it. A draft lands in a review folder, and it only enters the publish queue when I approve it. This is the one step I refuse to automate, because a structural check can catch a leaked IP but it can't catch a subtly overstated claim. A ten-second read by the person whose name is on it can.

Only after both gates does a draft join the queue.

Stage 4 — a boring publisher ships one a day

The publishing itself is the dumbest part on purpose. A scheduled job wakes up
once a day, takes the next approved post from the queue, publishes it, and moves
on. If a publish fails, the post stays in the queue and tries again tomorrow — it
never double-posts, and it never publishes something that isn't there. If the queue
is empty, it does nothing.

That "does nothing" is a feature. The whole system fails toward silence, never
toward fabrication. The worst case is that I post less often. The worst case is
not that a robot invents a week of accomplishments to keep the streak alive.

The ethics bit, because it matters

Every post this produces carries a visible line saying it was drafted with AI
assistance from my own notes and then reviewed before publishing. I'm not going to
pretend a human typed every word — that would be its own kind of lie. What I can
honestly claim is that every fact is real, gathered mechanically, and that a
human read it before it went out. "AI-assisted, human-verified, honestly labelled"
is a position I can defend. "Secretly generated, possibly fabricated, passed off as
hand-written" is not, and the pipeline is designed so I never drift into the second
one by accident.

Proof: how this exact post was made

Fitting, given the subject:

The radar flagged that I'd been building a content pipeline — from the real commits, not from a prompt.
The model drafted it from those facts, in my voice.
It passed the leak gate (no infrastructure details leaked — you'll notice I've named no IPs, no hostnames, no specific tools).
I read it, edited the bits that were slightly off, and approved it.
The daily publisher put it live, with its AI-assistance line at the bottom.

The system documents the system. That's either delightfully recursive or slightly
cursed, and honestly it's both.

What to steal

You don't need my code. The shape is the point:

Gather facts with a script; let the model only phrase them. The single most important line in the whole design. It's the difference between a writing assistant and a fabrication machine.
Derive topics from things that provably happened, so you can't write about work you didn't do.
Two gates: a machine one for leaks, a human one for judgment. Automate the check a computer does better; keep the check a person does better.
Fail toward silence. An empty queue posts nothing. Never build a system whose failure mode is inventing content to avoid a gap.
Label it honestly. AI-assisted and proud of it beats secretly-generated and exposed.

The reason I can post most days isn't that I found a robot that writes my blog. It's
that I built a machine that turns my real work into drafts, keeps the AI on a very
short leash, and lets me stay the editor. The cleverness writes. I still decide
what's true. That division is the whole thing.

Written by the pipeline it describes, reviewed by the human it describes, and
posted by the boring scheduler it describes. Turtles all the way down.

ZimaBlade as a real home server (not a NAS appliance): Docker, disks and what to change

Christian Anderson — Thu, 17 Sep 2026 08:00:04 +0000

I bought the little ZimaBlade the way most of us buy homelab hardware: convinced it would be a tidy appliance I'd plug in and forget. A fanless board, a NAS-oriented OS, a photo library and a couple of containers, done.

Three weeks later I was rooting around in Docker containers to borrow a root shell, because the thing I actually needed — my TLS certificates — had quietly fallen out of every backup.

This post is the honest account of that, in the form of two recipes I've now verified enough times to trust. They're living in a public repo: https://github.com/casareanderson/zimablade-recipes. The specifics are ZimaOS-flavored, but the two lessons transfer to any small server, and honestly to any server at all:

When your platform refuses to give you root, the container runtime is right there, already privileged. Borrow it.
Backing up "the data" is not backing up the system. If your certs, definitions, and config live outside the data, your recovery plan has a hole the size of everything you actually depend on.

Let me show you what broke, and what I changed so it wouldn't break again.

What the appliance assumption gets you

ZimaOS is clever about being approachable. Apps get installed from a storefront, they get their own little sandboxed runtime, and the UI tells you everything is fine. That's the whole promise of the form factor: you never need to see a terminal.

The trouble: an appliance treats you like a guest. Some operations — the ones that touch the OS itself, or that need a host-level privilege — simply aren't offered in the UI. There's no "give me a root shell" button, because the product assumes you'll never want one.

I wanted one. My setup needs to read some system state and reach interfaces that the sandboxed app runtime deliberately keeps out of reach. Nothing malicious — the kind of thing a normal Linux admin does without thinking — but the platform had no sanctioned path for it.

I spent a frustrating evening looking for the "correct" way. There wasn't one. Which is the first honest lesson: when a platform assumes you're a guest and you actually need to be an operator, don't burn the whole night searching for a supported path that doesn't exist. Look for the seam that's already there.

The seam: the container runtime is already root

Here's the thing about any of these little appliance-boxes running an app sandbox: under the hood, the apps are containers, and containers are just processes. Somewhere on that board there is a container with root privileges — your app containers almost certainly run as root inside their own sandbox by default. If you can get a shell into one of them, you have effectively borrowed root.

The recipe is short. Get a shell inside a running container — most of these runtimes let you do that from the UI or a shell command — and from there you can mount, inspect, and reach the host pieces you need. The container's boundary is exactly as high as you think it isn't.

Two things that made this work reliably for me:

Read-only first. Before changing anything, use the borrowed root to look. What's actually mounted on the host? Where are the app definitions? What has write access to what? You'll learn more about your own box in ten minutes of read-only poking than in a month of UI clicks.
Write down what you did. The whole reason this lives in a repo instead of my memory is that I forget shell one-liners. Every command I actually use to borrow root, and every command that depends on it, is now a numbered recipe so I can reproduce it after an update changes something.

The lesson that transfers beyond this box: when you're locked out of a system you own, look through the layer that's already entitled. It's usually the container runtime, and it's usually a few minutes of work to reach the access that is rightfully yours, to your own hardware, for the purpose you bought it for.

Recipe 02: backups that actually include your certificates

The second thing that broke taught me more, because it failed slowly and silently.

I set up what I thought was a solid photo-library backup. The library lives on the ZimaBlade; a scheduled job pushes it offsite to a low-power box at a relative's place, over an encrypted connection, starting with the originals. That part worked, and still works.

The failure was everything around the photos.

TLS certificates, for one. My offsite photo job, like most, transfers application data — the files the app owns. It does not transfer the platform's identity material: the certificate store, the keys, the trust configuration that lets my other devices authenticate to this box without heaving a warning every time. Rebuild that little board from scratch and the data comes back, but the TLS identity is gone, and reinstating it is not a button press. It's a day of regenerating, re-importing, and re-distributing trust — during which every client that talked to this device is throwing TLS errors.

And not just certificates. I realized my backup schedule covered the data the apps produced, but nothing about the definitions of the apps themselves. Recreate the box, and you've got the photos but you've lost the runnable setup — the storefront installs, the config, the file layout — that made the box what it was. Rebuilding that by hand is exactly the soul-destroying work I was trying to avoid by having backups.

So recipe 02 became two rules, and both are now encoded in the repo:

Back up the certificates alongside the data. Yes, treat secret material with appropriate care about where it lands and who can read it — but "too sensitive to back up" is how you end up with no recovery path at all. Encrypt it, restrict it, and ship it. The alternative is a box whose data you can restore but whose identity you can't, which is a different, worse kind of broken.
Back up the app definitions, not just the data. If you can't recreate the application layer from a list of what's installed and how it was configured, your backup is a partial backup. The photos coming back means nothing if the setup that served them is gone.

The general form of the lesson, and the one I actually want you to keep: a backup is only as good as the smallest thing you depend on that it doesn't include. Most of us design backups around the data we love — the photos, the library, the documents. But the things we truly depend on to run are often smaller, uglier, and easier to miss: a key, a config file, a manifest of installed apps. Data recovery without identity and definition recovery is a museum piece, not a working server.

Why they live in a repo

Everything above has the same shape: an observed problem, a command that fixed it, a note about where it broke. That's a recipe. And recipes rot if you don't keep them.

A few conventions I've settled into, because they're what made the difference between "notes I trust" and "notes I'd have to re-derive":

Genericize the site-specific details. Early drafts of my recipes were full of the specific hostname, the specific path, the specific network names for my setup. I stripped all of that out and replaced it with placeholders. Now the recipes are comprehensible to anyone, and — just as important — they force me to think about why a step exists, not just copy it.
Cross-link to related recipes. The GPU-report note and these two recipes reference each other, so a future-me landing on one page finds the other two things I also need.
Write down submission notes. When I post a recipe publicly, I note where it was shared and why, so months later I can tell which fixes actually helped people and which were just me.

None of this is glamorous. It's the "here's what worked AND what broke" record-keeping that makes tinkering with a tiny server feel less like tinkering and more like engineering.

The version I'd tell myself at the start

If I could hand a note to the version of me that unboxed this board, it would say three things.

First, an appliance is a marketing stance, not a property of the hardware. The moment you need one thing that the UI doesn't offer, you're a server operator whether you signed up for it or not — and the container runtime is ready to give you the access the menu won't.

Second, back up the whole system you depend on, not just the data you like. The certificates and the app definitions are not optional extras; they're the difference between a restorable box and a shiny paperweight.

Third, write it down the minute it works. Reproduce it, genericize it, and put it somewhere you'll actually find it again. Future you is a worse archivist than you think.

The recipes are public — borrow the root access technique, check your own backup for the missing certificates — at https://github.com/casareanderson/zimablade-recipes. Contribute back if you find a step that's wrong on your board; that's the whole point of sharing them.

It's a server. They just hadn't told you yet.

Free LLM API endpoints: a radar that finds them and a five-part gate before adopting one

Christian Anderson — Wed, 16 Sep 2026 08:00:03 +0000

Free LLM endpoints appear and vanish constantly — a provider opens a generous
tier, a new open model shows up on someone's gateway, a promo runs for a month.
Chasing them by hand is a losing game. So I built a small radar
(model-radar) that watches for
free models and, when one is genuinely good enough, adopts it automatically into
my agent stack — but only after it clears a gate strict enough that it has never
once adopted something that embarrassed me.

The auto-adoption is the flashy bit. The gate is the bit worth your time, because
"just use the free model" is advice that will quietly wreck an agent pipeline.

Why free models are a trap you still want to fall into

The appeal is obvious: a lot of agent work — summarise, triage, draft, classify —
doesn't need a frontier model, and paying frontier prices for it is silly. A free
tier that's "good enough" for the boring 90% is real money saved.

The catch is that "free model" and "works in an agent" are almost unrelated
properties. A model can score beautifully on a leaderboard and be useless for
agentic work, because agent work needs things benchmarks don't measure:

Reliable tool-calling. The model must emit correct, structured tool calls — and, more importantly, actually use the results it gets back. Plenty of models emit a tidy call and then ignore the response, which is worse than not calling the tool, because it looks like it's working.
Honest context handling. Some endpoints silently truncate a long prompt and answer from the half they kept, with no error. Confident, wrong, and invisible.
Stability. A free tier that rate-limits mid-task or disappears next week can't be in your critical path.

So a radar that adopted models on quality scores would be a machine for
introducing silent failures. The whole design problem is: how do you let a
process adopt something automatically without letting it adopt something bad?

The answer: adoption is a gate, not a vote

The radar can find candidates freely. It can promote one only by passing every
stage of a fixed, deterministic gate — and if any stage fails, it's rejected, full
stop. No score, no averaging, no "it's mostly fine." The gate checks, in order,
things a leaderboard never will:

Does it actually respond, at the endpoint, right now, within a sane timeout?
Can it tool-call for real — emit a correct call and incorporate the result into its next turn — on a concrete test task, not a synthetic one?
Does it respect context — feed it something long and confirm it didn't silently drop the tail?
Is it stable enough across repeated calls, not a one-lucky-response fluke?
Does it beat what I'm already running on the specific jobs I'd actually route to it?

Only a model that clears all five gets adopted. Everything is measured on real
tasks and written down, so adoption is a reproducible decision I can audit later,
not a vibe. And there's a blocklist: models measured to fail agentic use in a way
that matters are banned by name, so the radar can't rediscover and re-adopt a known
dud.

The principle underneath is the same one my whole estate runs on: let the
automated part gather and propose freely; let it act only through a gate that a
bad candidate cannot argue its way past. A guardrail written as a suggestion in a
prompt is decoration. A guardrail written as a pass/fail function is a wall.

What it actually buys me

Most of my agent work runs on free or cheap models, adopted automatically as better free options appear, with the expensive frontier models reserved for the genuinely hard slice.
I'm not manually chasing which provider opened a tier this week.
And nothing junk gets in, because adoption is gated on the properties that actually matter for agents — tool-calling, context honesty, stability — not on a benchmark number.

The failure I designed the whole thing to avoid is the one that gets everyone else:
a cheaper model quietly swapped in that looks like it's working — emits calls,
returns fluent text — while ignoring tool results or truncating context, producing
confident garbage that no error ever flags. The gate exists specifically to catch
the model that fails silently, because that's the only kind of failure that
survives long enough to hurt you.

Take a look

model-radar is on GitHub. Even if
you never auto-adopt anything, the transferable idea is the gate: before you route
real work to a free model, test it on the thing agents actually need — can it
tool-call and use the result, does it keep your whole prompt, is it stable — and
make that test a hard pass/fail, not a score. Cheap intelligence is worth
chasing. Cheap intelligence that fails silently is worth fearing, and the only
difference between the two is whether you gated it.

Notes from someone who wanted a coffee-a-month agent bill and refused to pay for
it in silent failures, so built a radar that adopts free models automatically and a
gate strict enough to trust it.

Random song picker for Spotify: a physical-feeling 'jukebox dice' for 6,500 tracks

Christian Anderson — Tue, 15 Sep 2026 08:00:04 +0000

Choosing what to play is a solved problem right up until it's your turn to DJ at a
family thing and everyone's watching. So I built the dumbest, most fun fix I could:
spin-that-dice — a jukebox
die you tap, and it picks a random track from a curated crate of thousands and
just plays it. No scrolling, no "what should I put on," no dead air. Roll the dice,
music happens.

It's a small, silly project. It's also taught me two genuinely useful things — one
about designing for delight and one about a nasty API failure mode — so here's
the tour.

What it is

A crate of ~6,500 tracks, sorted into dozens of categories — hip hop, dancehall,
afrobeats, Motown, soul, grime, amapiano, and a couple of dozen more. You "spin the
die," it lands on a category, pulls a random track from it, and starts playback on
your own account. The whole interaction is one tap and a moment of suspense, which
turns out to be the entire point.

The design goal wasn't "a better music browser." It was to remove choice as a
source of friction and add a tiny bit of theatre. A playlist makes you a curator;
a dice roll makes you a participant. At a party, the second one is far more fun, and
"what's it going to land on?" does more work than any recommendation algorithm.

The small design lesson: constraints are the feature

The temptation with a project like this is to add: filters, history, skip-weighting,
mood detection. I kept saying no, and the tool got better every time I did.

The magic is in the constraint. You don't get to fine-tune it. You roll, you
get what you get, you either vibe with it or roll again — and that little bit of
surrender is exactly what makes it feel like a jukebox instead of a search box. Once
I understood that the randomness was the product, the rest of the design wrote
itself: make the roll feel good, make the crate deep enough that repeats are rare,
and get out of the way.

If you build tools, it's a useful reminder: sometimes the feature is the thing you
refuse to let the user control.

The gnarly bit: a rate limit that re-arms while you wait

Now the war story, because it's the transferable part.

The tool drives playback through a third-party music API, and that API rate-limits.
Fine — everyone rate-limits, you back off and retry. Except this one had a trap that
turned a polite retry loop into a permanent lockout.

The failure: when I got throttled and backed off, the throttle window re-armed
every time I probed it. So my well-behaved "wait, then check if I'm allowed yet"
loop was itself the thing keeping the door shut — each check reset the timer I was
waiting on. A retry strategy that works against a normal rate limiter (poll until
it clears) was exactly wrong here: the polling was the problem, and the harder I
tried, the longer I was locked out.

The fix was counterintuitive: back off longer and probe less, not more. Once
I stopped hammering the check, the window actually expired and playback resumed. The
lesson generalises past this one API:

There was a second, dumber gotcha too: a LAN hostname for the thing needs a real DNS
record, not just a local rewrite — but that's a story every self-hoster already
knows in their bones.

Steal it, or the idea

spin-that-dice is on GitHub. If
you want a zero-friction party player for your own crate, clone it and point it at
your account. If you just want the idea: the next time you're building a
"choose from a big list" interface, ask whether the choice is actually the fun
part — or whether a dice roll would delight people more than a filter ever could.

And if you're building anything on a rate-limited API: find out whether its limits
clear on a timer or re-arm on contact before you write your retry loop, because
those two worlds want opposite strategies, and guessing wrong locks you out of your
own party.

A genuinely daft little tool that's more fun than it has any right to be, plus one
rate-limiting scar I'm still slightly annoyed about. Both real, both on the account
I actually DJ from.

DEV Community: Christian Anderson

My homelab agent can only recommend what the dataset can prove

What I Built

Demo

Code

How I Used Sanity

Sanity Project Details

Ollama keep_alive -1 hangs every other model: a silent deadlock on a 6 GB GPU

The first time: 26 August

The fix that wasn't

The second time: 10 September

What's actually happening

It comes back within hours

The actual fix: a guard, not a setting

What I'd tell anyone running Ollama on a small card

Cryptonym Desk on Sanity: moving a word list into a dataset found two bugs

What I Built

Demo

Code

My Build Process

Update: word proposals, a workflow kept as data

Sanity Project Details

Agent Session

Piper TTS over Wyoming: the frame format and empty model that broke my podcast

The shape of it

Voice: Piper over Wyoming, no new service

Bug one: the frame has three parts, not two

Bug two: the model that answered in a thinking key

The small 7B problems

The rule that keeps it honest: the source sets the ceiling

The mix

The licence check

What I'd tell you if you're building one

Authelia passkeys with TOTP as break-glass: why the config diff was empty

What the policy actually says

Why the passkey is the normal one now

Why TOTP is still enrolled

Where 2FA can't go

The takeaway

eBay sold prices without the paid API: I built my own sold-comps engine and measured it

The paid meter was running out

The premise (and the thing that made it possible)

How it works

What the numbers said after a week

The things that bite (four of them)

What the GitHub repo gives you

The takeaway

Naming AI agents: what the CIA's declassified cryptonym convention taught me

This is not a market gap

Three names, because one name is wrong

The alias is a mashup, and the seams are ugly on purpose

The records are seeded, and everyone finds this weird

The deployment trap: an artifact is not a web page

What it's made of

Spotify + djay on an iPad as a hands-off party DJ: my child's baptism, and an agent that ignored the brief

The brief

Attempt one: the agent ran the wrong job

Attempt two: rebuilt from the brief, by hand

The Spotify API fights back

The playback: djay on an iPad, hands off

What Hermes actually did, and didn't do

AI blog writing pipeline without made-up facts: how I post daily from real work

The problem with "just get AI to write your blog"

Stage 1 — a deterministic radar finds real topics

Stage 2 — the model drafts, from those facts only

Stage 3 — two gates before anything is public

Stage 4 — a boring publisher ships one a day

The ethics bit, because it matters

Proof: how this exact post was made

What to steal

ZimaBlade as a real home server (not a NAS appliance): Docker, disks and what to change

What the appliance assumption gets you

The seam: the container runtime is already root

Recipe 02: backups that actually include your certificates

Why they live in a repo

The version I'd tell myself at the start

Free LLM API endpoints: a radar that finds them and a five-part gate before adopting one

Why free models are a trap you still want to fall into

The answer: adoption is a gate, not a vote

What it actually buys me

Bug two: the model that answered in a `thinking` key