DEV Community: Chad Priest

Your MEMORY.md is a file nobody writes to. Render it instead.

Chad Priest — Thu, 27 Aug 2026 06:57:50 +0000

Every agent harness injects a memory file at session start. Claude Code has MEMORY.md. OpenClaw has a workspace MEMORY.md. Starpod keeps one per user. The file is the one thing the model reads on every turn, so it is the highest-leverage artifact in the whole memory stack. And in most systems it is hand-edited, which means it is maintained by whoever remembers to maintain it, which means nobody.

I found this out by counting. My memory store held 42,000 chunks. The machine-managed zone of the injected file had received one bullet since 2026-04-08. Four months. One bullet. The retrieval pipeline was working; the artifact the model actually saw was frozen.

One bullet in four months against 42k chunks in the store

The fix was to stop treating the file as a source and start treating it as a view. MEMORY.md is now rendered from the database at session start, per session and per project, by a deterministic renderer.

The rendering has three tiers. Pinned chunks go first, under Identity, Preferences, Decisions and Notes, and they appear in every project. Then the memories tagged with the project the session is running in, resolved from the working directory by the same longest-root-path rule the prompt hook already used. Then global memories fill the remainder of a 6 KB budget (later raised to 8 KB), restricted to durable tags or importance 7 and above. Imports and raw captures are excluded from the render entirely; more on why below.

The file is a view, not a source

The file on disk still exists, because two readers (the workspace loader and the gateway) read it without talking to the daemon. The daemon rewrites it every 60 seconds, atomically, and the write is a no-op modulo the timestamp line if nothing changed. If the daemon is down at session start, the snapshot stands. If it is up, the hook gets a per-project rendering and splices it into the injected section.

For a person using it, the change is one rule: never edit the file. To change what every session sees, pin a chunk or pin a sentence into a section. Everything else in the file is earned by importance, tag and recency, and it moves.

74 of 42k chunks carried a project id, so the project tier was empty

The renderer worked on day one and the project tier was thin to the point of uselessness. The reason was not the renderer. Only 74 of 42,000 chunks had a project_id. The tier had nothing to select.

The evidence to fix that had been on disk the whole time. Every Claude Code transcript line carries cwd. Nothing had ever read it. So the IDE capture path now resolves a transcript's dominant cwd to a project and stamps the conversation, and the extractor, which already wrote project:<id> on bullets from tagged conversations, needed no change. A backfill pass over 489 transcripts on disk matched 341; three were tagged to a real project and 338 resolved to the install root, which is global by design. Historical chunks were not retagged, because there is no chunk-to-conversation link, and I rejected keyword tagging outright: a wrong project tag hides a memory from every other project's recall, which is worse than no tag.

Then the stamp itself went wrong. extract_project trusted any project: token found in text, so a plan document that merely mentioned the token got stamped with whatever followed it. Seven live chunks carried ids like <id>, changed and \S+\s+)?. A junk stamp is strictly worse than none: a non-NULL project_id is neither global nor in any real project, so the chunk falls out of the rendered file, out of the project boost in search, and out of every vault carrying a project rule. My first guard pinned the id to exactly eight hex characters, and three existing tests failed because they use proj_abc fixtures. That was the useful failure. An over-strict validator drops a legitimate stamp, which does the identical harm to the junk it stops. The shipped guard checks shape (the proj_ prefix), which all seven junk values fail on their own.

Zero of 42,798 chunks had a valid_at, and imports outranked native facts

The bigger dead end was time. The renderer and the search ranker both sort on recency. Recency was created_at. created_at answers when a row was written, not when the fact was true, and my temporal-memory plan had specified a valid_at column for exactly that distinction. The column shipped. The writer never did. I audited on 2026-08-16: 0 of 42,798 chunks had one.

Two defects had been hiding inside that zero. First, import collapse: an archive is written all at once, so all 3,403 ChatGPT import chunks, three and a half years of history, landed inside one 24-hour window. Second, re-sync re-dating: re-indexing a file deletes and reinserts its chunks, and created_at was not in the preserve map, so every chunk of memory/2026-05-01.md read 2026-05-02 03:56, the following night's pass. No chunk's date matched its own filename.

What the audit found on 2026-08-16

The live cost was measurable: 87 rows where an import chunk held canonical over a native one, because the inject tie-break ("yesterday's correction beats last year's stale fact") was reading last year's fact with yesterday's timestamp. This is why imports and captures are excluded from the rendered file. They are not less true; their clocks were lying.

Writing valid_at was not enough on its own. The reconciler, which decides whether a new fact supersedes an old one, already had a deterministic guard that refuses to move memory backwards in time. The guard stopped the damage but taught the model nothing: the LLM kept proposing updates the guard silently refused, and both facts stayed standing side by side. The prompt showed no dates, so "NEW" read as "newer" when it only meant "newly ingested". Each fact in the prompt now renders a [YYYY-MM-DD] from valid_at (else created_at), an explicit [undated] where there is none rather than an empty bracket the model could read as a parse failure, and the instruction says to choose UPDATE only when the new fact's date is the later one.

The invariant: every column a ranker sorts on has exactly one writer that means it

Here is the class, stated so you can check it. For every column that participates in ranking, selection or supersession, there must be a writer whose semantics match the column's name, and the writer must run on every ingest path. valid_at failed the first half (no writer). project_id failed the second half (one path wrote it, one path wrote garbage, most paths wrote nothing). In both cases the ranker was correct and the column was hollow, and a hollow column does not error. It sorts. It just sorts on the wrong thing, and the artifact the model reads looks fine.

The corollary is that the injected file cannot be a source of truth, because a file has exactly one writer (a human) and that writer's semantics are "when I remembered to". Make it a view and the columns become the only thing you have to get right.

Three queries to run on your own store

You need nothing from Vodou for this. Point them at whatever SQLite or Postgres table holds your memories.

First, does your ranking column have a writer? Substitute your event-time column for valid_at.

SELECT COUNT(*) AS total,
       SUM(CASE WHEN valid_at IS NULL THEN 1 ELSE 0 END) AS missing
FROM chunks;

Passing looks like total=42798, missing=120, a small residue you can explain. Failing looks like missing equal to total. If it does, every "recent" boost in your ranker is a write-time boost, and anything you bulk-imported is winning ties it should lose.

Second, are your creation timestamps clustered where they should be spread?

SELECT date(created_at) AS day, COUNT(*) AS n
FROM chunks
WHERE source LIKE 'import:%'
GROUP BY day ORDER BY n DESC LIMIT 5;

Passing is a spread across many days. Failing is one row holding most of the count. For me it was 3,403 chunks on one day.

Third, is your scoping column populated, and populated with values that resolve?

SELECT project_id, COUNT(*) FROM chunks
WHERE project_id IS NOT NULL
  AND project_id NOT IN (SELECT id FROM projects)
GROUP BY project_id;

Passing is zero rows. Failing is any row: those chunks are in no scope at all, invisible to global and to every project. Then check the ratio of non-NULL to total; mine was 74 over 42,000, which is not a scoping feature, it is a scoping placebo.

And one check for the file itself, if you inject one. Diff its machine-managed zone against a copy from a month ago. If the diff is empty and your store grew, the file is not a memory. It is a fossil.

What the memory architecture guides skip: the file is the read side

The published patterns are good about layers and poor about the artifact the model actually reads. The agent memory pattern spec says to score retrieval by recency, frequency and relevance, and never says which timestamp recency means; mine was the wrong one for a year. The engineering playbook splits long-term memory into episodic, semantic and procedural, which is a fine taxonomy for storage and silent on how any of it reaches the context window on turn one. INOSX's five-layer design is the closest, with an explicit "read-time assembly" plane, which is the right framing: the injected file is an assembly, and assemblies are rendered.

The tools that do keep a MEMORY.md mostly index it. Gigabrain syncs the workspace MEMORY.md alongside its registry for unified recall, and sae4u-memory searches SQLite facts and the markdown auto-memory dirs in one call. Both treat the file as a second corpus to read from. I went the other way: the file is an output of the database, never an input. hermes-memory is nearest in spirit, keeping the hot injection under about 180 tokens and migrating accumulated facts out of the file into the DB, which is the same instinct with a smaller budget. My hot tier is 6 to 8 KB because the pinned identity and preference facts are the part that must never be lost to retrieval scoring.

Still open: historical chunks keep their project blindness

The backfill stamped conversations, not chunks. The 42,000 chunks that existed before 2026-08-16 have no link back to the conversation they came from, so they will never move into a project tier unless they are re-extracted. I chose that over keyword tagging because a wrong tag is a hidden memory, but the honest state is that the project tier is accurate only for memory captured after the change. Whether the store's older half is worth re-extracting against the corrected paths is a measurement I have not taken.

Source: Your MEMORY.md is a file nobody writes to. Render it instead. by Chad Priest, from Building Vodou in Public.

A 400-byte cap crashed my memory daemon on one emoji

Chad Priest — Thu, 27 Aug 2026 06:51:48 +0000

Every agent memory system trims text somewhere. Facts get capped before they go into a prompt. Summaries get cut to fit a budget. Recall blocks get sliced to a length. The cut is usually one line, written months ago, and it takes a number that somebody assumed was characters. It was bytes. Or UTF-16 code units. The first user who puts an em dash or a party emoji near the limit finds out which.

That is one of two bugs I hit this week while moving Vodou's memory graph into the console so that memory is one store served by one process. The other one is about a guard that ran perfectly and left no trace. They have the same shape, and that shape is the point of this post.

The move: one graph, one process, and 49,338 chunks that had never been deduplicated

The capability itself is unglamorous. The brain graph (the map of facts, their groups, and which twin retired which) used to live in a separate process with its own database handle. It now lives inside the console, reading through the same API as the memory list, the vault rules and the document library. The public side is in MCP-servers/Vodou-Console/src/brain/queries.ts and src/api/brain.ts, with a drift test in src/__tests__/brain-queries-drift.test.ts so the graph queries and the list queries cannot quietly disagree about what a live chunk is.

For a person using it: one search box, one theme, one process to restart, and a map whose filter searches memories instead of filenames (that one was a bug too). The engine underneath it gained fact groups and a reconcile step that decides, when two facts say the same thing, which one lives.

Where a fact goes before it is a fact

Getting the graph to tell the truth meant fixing what it displayed. Three of those fixes are worth your time.

Exit 99, and the hook said "memory DEGRADED" instead of "the daemon is dead"

Reconcile runs over every fact it compares, and it trims long facts to 400 for display in the prompt. The cap was written as a byte length, and the standard library truncate in that language asserts when the index lands inside a multi-byte character. A fact with an accent, an em dash or an emoji near position 400 took the daemon down with exit 99.

What made it expensive was not the panic. It was the symptom. The worker process survived, so from the outside memory looked empty rather than crashed. The prompt hook reported DEGRADED and connection refused. I spent time on sockets while assertion failed: is_char_boundary sat in the system log. The bug had been latent since 2026-07-16.

The fix is one line: cut on a character boundary, and make the cap 400 characters, which is what the call site always meant. It was trimming for display, not enforcing a byte budget. The regression test fills the string with 2-byte and 4-byte characters, both of which panic under the old code.

This is not a Vodou bug. It is the industry's bug this year. OpenFang hit it at a fixed byte offset with an em dash. AionUi hit it on a 25,000-byte MEMORY.md cap with CJK text, where the panic killed the agent at session start and every later start too. Mastra hit the UTF-16 version: truncation split a surrogate pair and Anthropic's JSON parser rejected the whole request. OpenClaw shipped the same fix for recall summaries prepended to the prompt. Four projects, four months, one line each.

2,437 backwards clocks, and the reconciler that kept the poorer twin

The second fix started as a user-visible failure: Vodou had learned my sons' names four times and thrown them away four times, keeping "married, two sons" every time. Chasing it found two unrelated bugs wearing one symptom.

The first was inert. 2,437 of 3,281 retired chunks had a retirement time earlier than their creation time. Re-syncing a memory file deletes and re-inserts its chunks, so creation is re-stamped from the file while retirement is carried over from the deleted row. Two columns, two sources, no invariant. I backfilled them and then checked all 33 readers of that column: every one tests for NULL, none compares it as a time. So it changed nothing anyone saw. I recorded it anyway, so nobody hunts for a symptom that does not exist.

The second one bit. Reconcile's duplicate arm was first-writer-wins. The incumbent survived, the newcomer was hidden. A later, richer phrasing lost purely for arriving second. The fix retires the fact that says less, not the one that arrived later.

Underneath both sat a third problem: near-duplicate demotion had never completed on this corpus at all. The scan is O(n²) over 49,338 chunks, and the CLI watchdog (progress-based since 2026-08-22, so slow-but-advancing work survives) killed it every run with "no progress since start, 90s". A lane that never beats is indistinguishable from a wedged one. The fix was to beat every 512 chunks, not to add a 17th exemption. First completed scan: 428,484 pairs over cosine, 8,407 passing overlap, 980 groups, 3,292 demoted, 20 value conflicts queued, 147 seconds. Retrieval-bench moved with it.

retrieval-bench above-floor, before and after the first finished dedup

Zero verifier lines after 120 facts, and both explanations fit

The bridge's 4B extractor turned a turn that said "just saying hi" into a preference fact about reply length, and wrote it. One model read the transcript, asserted what it meant, and its assertion was the verification. Extraction now runs each candidate fact past a separate completion that sees only the source and the claim and answers one question: does the source say this. Not "is this plausible". Plausible-given-the-source is exactly the trap.

Then I went to measure the rejection rate. Five conversations, 120 facts written, zero verifier log lines. I spent several steps hunting a wiring failure that did not exist. The verifier had run the whole time. It only logged on rejection, so "ran and approved everything" and "never ran" produced byte-identical evidence: nothing.

I had built, into the instrument whose purpose is catching unchecked claims, a log that could not distinguish success from absence. It now emits one line per batch, unconditionally:

[extract-verify] mode=Log checked=4 kept=4 unsupported=0 unknown=0

An absent line now means the verifier did not run. unsupported and unknown are separate numbers on purpose: a high unknown means the checker could not reach its provider, and must never be read as "the facts were clean". First real rate from a live turn: 8 checked, 0 unsupported, 0 unknown.

The class: a cap in the wrong unit, and a guard whose silence means two things

Both bugs are checkable properties, not advice.

First: every truncation applied to user text must count in the unit the consumer of the text counts in. A prompt-display cap is characters. A storage cap is bytes. A JSON-over-HTTP cap must never split a UTF-16 surrogate. If the cap's unit and the cut's unit differ, the code is wrong regardless of whether it has fired yet.

Second: a guard that gates writes must produce evidence on every invocation, including approvals. If the set of observable outputs for "ran and passed everything" equals the set for "did not run", the guard is unobservable and its rejection rate is unmeasurable. This is the same property as a health check that returns 200 from a stranger's process.

Run this on your own store: an emoji at the cap, and a grep for silent guards

Find your cuts. In a Rust, Go, or C tree:

grep -rnE '\.truncate\(|\[\.\.[A-Za-z_0-9]+\]|\[:[A-Za-z_0-9]+\]' --include='*.rs' --include='*.go' src/ | grep -v test

In a TypeScript or Python tree:

grep -rnE '\.slice\(0, ?[A-Za-z_0-9]+\)|\.substring\(0|\[:[A-Za-z_0-9]+\]' --include='*.ts' --include='*.js' --include='*.py' src/ | grep -v test

Each hit is a cap. For each one, ask what unit the number is in and what unit the slice is in. Then feed the function a string that puts a 4-byte character exactly on the boundary. In Node, this is the whole test:

const cap = 5;
const s = "abcd🔥ef";
const cut = s.slice(0, cap);
console.log(JSON.stringify(cut), cut.isWellFormed());

Failing output is "abcd\ud83d" false: a lone high surrogate that JSON.stringify will happily emit and a strict server-side parser will reject. Passing output ends in a whole character and prints true. In Python, s.encode()[:cap].decode() raises UnicodeDecodeError on a failing cap and returns cleanly on a passing one. If your code path is Rust, use the char-boundary-aware cut and write the test with é and 🙂 fills; both panic under a byte cut.

For the second property, take your write-side guard (the schema validator, the PII redactor, the fact verifier, whatever stands between a model's assertion and your store) and run this against its log after real traffic:

grep -c 'verify' app.log; grep -c 'INSERT INTO facts' app.log

If the second number is large and the first is zero, you have one of two situations and cannot tell which. Passing looks like a guard line count that is at least the batch count, with an explicit approved count in each line. Failing looks like silence, and silence is what you saw before you looked.

What the memory guides say about consolidation, and the two things they skip

The memory-architecture literature is good on tiers. The Geodocs pattern spec lays out working, episodic, semantic and procedural memory with consolidation and eviction rules. The Engineering Playbook maps the same three flavours onto storage choices. Pockit's production guide is right that context windows are not memory.

None of them say who verifies the extractor. Consolidation is described as a step, not as an assertion that needs a second reader. And none mention that the prompt you consolidate into gets truncated somewhere, and that the truncation is where four projects fell over this year. Tiers are the easy part. The seam between "the model said this" and "the store now holds this" is where memory quality is decided, and it needs both a checker and a log of the checker.

Still open: the checker is graded on four cases, not on a corpus

The verifier has been checked against four discriminating cases and one live turn of eight facts. That is enough to know it runs and to know its log is honest. It is not enough to know its false-approval rate on a month of real transcripts, and "0 unsupported" on eight facts is not a rate. Until a proper labelled set exists, the number to watch is unknown, because that one tells me when the checker stopped being able to check.

Source: A 400-byte cap crashed my memory daemon on one emoji by Chad Priest, from Building Vodou in Public.

Your document chunker is a memory chunker, and it quadruples your store

Chad Priest — Thu, 27 Aug 2026 06:44:01 +0000

Every memory system I have looked at, mine included, grew a document ingest path by pointing the existing chunker at a folder. It splits on bullets, it caps at a few hundred characters, it has worked for months. Then a real directory goes in and the store gets four times denser, the router starts ranking a 0.000 above a 0.746, and nobody can say which of two scorers produced the number on screen. None of that is a model problem. All of it is a seam between two things that were quietly sharing one rule.

This is what shipped over eight days in Vodou's library lane, and more usefully, what was wrong first.

A library is not a memory: whole files, one card each, two ways to match

The capability is plain. You add a document from the Library page (a file, a folder, or a URL), it is extracted and indexed as a whole document, it gets a one-row "card" summarising what it is about, and it becomes addressable in any chat as an @doc: token. Two matching lanes answer two different questions. The subject lane asks "is this document what you are asking ABOUT" and scores with a cross-encoder. The topic lane asks "does this document DISCUSS what you asked" by finding the best passage in the chunk index and mapping it back to its file. The two are never sorted against each other, because a cross-encoder probability and a raw cosine are not the same kind of number.

Two lanes, two scales, never one sort

The public half is MCP-servers/Vodou-Console/src/api/library.ts (the routes), extension/Store-vodou-bridge/background.js (the URL and page-text lanes) and MCP-servers/Vodou-Console/src/__tests__/library-e2e.test.ts, which is the harness that should have existed first.

181 chunks from one 15,869-character file

The directory ingest command already existed. Its first real run turned a 15,869-character plan file into 181 chunks, about 88 characters each, against a 600-character ceiling. The reason was correct behaviour in the wrong place. For memory files, a bullet is a fact: "- The user's name is Chad." has to be its own retrieval unit. For a document, "- [ ] Run npm run build" was becoming its own embedding row. Across the 643 files I actually wanted in, that density would have roughly quadrupled a 38k-chunk store.

The engine now carries a chunk mode, memory or document, and the same file went from 181 to 50 chunks with nothing dropped. One test pins the difference between modes rather than either mode alone.

One plan file, two chunk modes

The first fix changed nothing, and that miss is worth keeping. I patched the function that applies the ceiling, which is the probe. The writer re-chunked in memory mode on its own. Probe and writer had two copies of the decision, so fixing one left the store exactly as dense as before.

0.70 was calibrated on cosine and applied to a cross-encoder

Then the router. The card lane had a 0.68 floor justified by a code comment citing one pair of observations. Measured against a 22-query labelled set, the relevant population (0.553 to 0.845) and the ordinary-browsing population (0.455 to 0.686) overlap outright. "Sourdough Starter Recipe" scored 0.686, above a genuine contract query, because a bi-encoder rewards surface overlap between a short title and a long summary. No constant separates those two populations. Switching the card scorer to the cross-encoder collapsed every noise query to about 0.000 and routed all 8 relevant queries to the correct document. It also took match from 0.36s to 8.3s cold, so a cosine gate below every floor skips the model for ordinary browsing.

That gate created the next bug. The score field was overwritten with a cross-encoder probability when the reranker ran and left as a cosine when the gate skipped it. The 0.70 floor was calibrated on cosine. So whether a document was eligible depended on whether a latency gate happened to fire. Three queries, each ranking the right document first under both scorers, each rejected under only one:

query                       cosine   cross-encoder
launching on hacker news     0.703       0.000
console redesign             0.720       0.001
go to market launch plan     0.676       0.052

Meanwhile sub-floor subject hits were never dropped, and because topic hits are appended after subject hits, a 0.000 non-answer printed above the 0.746 right answer. I first filtered them out, which turned "go to market launch plan" (right document, 0.052) into "no documents matched". Losing the answer is worse than mis-ordering it, so sub-floor hits now survive as a tagged weak fallback used only when neither lane produced anything real.

The split that stuck: cosine decides eligibility and is the number shown; the cross-encoder decides order among the eligible; score is restored to the cosine after reranking. Cosine alone gets "go to market launch plan" wrong (0.687 for the wrong plan). The cross-encoder alone rejects it. Each does the one thing it is calibrated for.

There was a third leak under load. The reranker had four early-return paths and only one was safe. When the model failed to load under memory pressure, raw cosine stood and "Getting Started with React" cleared a floor that means nothing on a cosine. Cosine ordering is a weaker signal; cosine thresholding is a broken one. The result now carries an explicit calibrated-or-not marker, and an uncalibrated result is suppressed rather than answered. On a machine with no inference runtime the lane goes silent and says so. All 11 noise queries: 0 leaks, from 1 to 2 under load.

The failure class: one field, two producers, one floor

Two invariants fell out, and both are checkable against a codebase rather than advice.

First: the chunk-size decision has exactly one owner, and the probe that predicts the chunk count and the writer that produces it call it. If they can disagree, the ceiling trims the wrong amount and a test on the probe passes while the store fills.

Second: a threshold is bound to the distribution it was calibrated on, so any field a floor reads must have exactly one producer or a calibration tag. If a rerank step can be skipped, disabled, or fail, and on every one of those paths the same field holds a different scorer's output, then eligibility is a function of latency and memory pressure, not of relevance.

Two queries to run on your own store tonight

Both take five minutes on any SQLite or Postgres chunk table. First, chunk density per source. Documents and facts should not share a distribution.

SELECT source_id,
       COUNT(*)                          AS chunks,
       SUM(LENGTH(content))              AS chars,
       SUM(LENGTH(content)) / COUNT(*)   AS avg_chunk
FROM chunks
GROUP BY source_id
ORDER BY chunks DESC
LIMIT 20;

Passing: your long documents sit near your configured ceiling (a 600 ceiling should show 400 to 600). Failing looks like mine did, 181 | 15869 | 87 for a single file. If your average chunk is a fifth of your ceiling, your document chunker is a fact chunker.

Second, the scale check. Log the retriever's final score alongside which scorer produced it, then look at the floor's neighbourhood.

SELECT scorer, COUNT(*),
       MIN(score), AVG(score), MAX(score)
FROM retrieval_log
WHERE score BETWEEN 0.60 AND 0.80
GROUP BY scorer;

Passing: one row, or two rows whose ranges do not both straddle your floor. Failing: cosine averaging 0.71 and cross_encoder averaging 0.03 in the same band, which means the same floor is admitting one and rejecting the other by accident. If you have no scorer column, that is the finding: add it before you tune the threshold again.

The retrieval-quality posts say measure recall; the seam is inside one score

The current writing is right that context-building fails before generation does. The New Stack's piece on retrieval as the defining bottleneck and Weaviate's retrieval-quality overview both land on the same point: when retrieval breaks, the model extrapolates with full fluency. Shaped's stochastic parrot post names the semantic trap where a vector store finds fifty near-misses. What none of them say is that the trap can be a data-shape bug: a single score column with two producers, or a chunker with two callers. Recall benchmarks report the symptom. The mem.nowledge library docs get the product split right (documents stay whole, memories are takeaways), and that split has to reach the chunker and the scorer too, not stop at the UI.

Still open: the topic lane leaks on vocabulary, and there is no number

The topic lane matched react.dev pages to implementation plans and doc.rust-lang.org to anything with a Rust paragraph. Two threshold calibrations and a cross-encoder swap failed: true interior hits sat at 0.702 to 0.741, false ones at 0.703 to 0.743, interleaved point for point. The fix that held is a document-level veto (the cited passage's heading must mention what was asked), 3 of 3 false positives rejected, 0 of 4 true hits killed. But that labelled set is 22 queries on one library, and the subject lane still scores real paraphrases near 0.0, so weak ordering is barely better than chance. The e2e suite reports the topic leak count on every run instead of thresholding it, because a cap calibrated on a live library breaks within hours. I have the number for mine. I do not yet have one I would trust on yours.

Source: Your document chunker is a memory chunker, and it quadruples your store by Chad Priest, from Building Vodou in Public.

Per-site memory off switches only work if every reader asks the same authority

Chad Priest — Thu, 27 Aug 2026 06:36:19 +0000

If your agent reads the browser tab, you already have a policy problem, whether or not you have written the policy. The extension sees a bank login page the same way it sees a Wikipedia article. The memory store does not know which facts came from a health portal. And the day a user asks "forget everything you learned from that site", you find out how many code paths actually read that data, because each one needs to honor the answer.

I shipped the governance layer for Vodou's page memory across two days in August. This is what it took, including the parts the tests said were fine and were not.

Off, suggest, collect: the mode lives in the gateway, not the extension

The capability is small to describe. Every host gets a mode: off, suggest, or collect. Banks, health portals, tax sites and sign-in hosts default to off, which means the extension never reads the tab there until the user flips it. Everything else collects. Precedence is one line: a user rule beats the sensitive default, which beats the global default, and a rule covers subdomains.

The part that matters for your stack is where the mode is decided. It is not in the extension. The gateway is the authority, exposed as GET/PUT /api/page-match/site-mode and GET /api/page-match/site-modes in MCP-servers/Vodou-Console/src/page-site-mode.ts. The extension asks before it does anything with a page. Typing suggestions ask the mode first and send nothing until it answers. That ordering is the whole design: the client is not trusted to remember the policy, it is required to fetch it.

One authority, three enforcement points

Three paths read page memory: the match endpoint that fills the side panel, the probe that colors the toolbar icon, and the note and link routes that write a fact with a page attached. All three now check the mode. Off means the match returns empty without asking the daemon at all, the probe returns no icon, and a write returns 403 unless the mode is collect. Those are three separate checks because they are three separate readers, and I tested them separately: four tests for the mode resolution, four for enforcement, in MCP-servers/Vodou-Console/src/__tests__/page-site-mode.test.ts.

Forget a host: soft delete, dry run first, confirm within eight seconds

The second half of governance is retraction. mem forget --host <h> sets invalid_at on every fact whose source host is the host or a subdomain of it. It does not delete rows. Library documents saved from that host are counted and reported, not removed, because a saved article is something the user chose deliberately and I did not want one command to take two kinds of data.

The panel exposes it as "Forget site…" with a two-click shape: the first click runs a dry run and shows the count, the second click within 8 seconds confirms. --undo clears the invalid_at stamps back. I verified the loop live on w3schools.com: forget hid one fact, the count read 1, undo brought it back, the count read 0.

Soft delete has a cost that a hard delete does not: every reader must filter on invalid_at. That is the same shape as the site-mode problem. A governance decision is only as good as the number of readers that honor it.

The unit suites were green and the live page found four defects

Before P4 landed I loaded build .74 unpacked and ran it against real pages: ChatGPT with the consent gate open in DevTools, a Wikipedia clip, a saved article, the Ctrl+Shift+B regression. Every unit suite had passed. Four things were wrong anyway.

A fact I had stored 30 minutes earlier displayed as "18h" old. The by_page query returns COALESCE(valid_at, created_at), and a fact that came out of a daily log has its valid_at set to the log day at local midnight. The age math was correct and the input was a day-granular instant. Day-granular instants now render as today, yesterday, or N days.

"Send selection to Vodou memory" sent no URL. The two Wikipedia clips I made landed with an empty source_url, which means they could never be found from that page again. The selection was the only content the handler passed along. It now sends the tab's URL and title as well.

The lane the "Add this page to Vodou Library" action actually uses, add_text, never wrote memory_sources.source_url and never stamped its chunks. Only add_url did. Two entry points to the same table, one of which stamped provenance. I mirrored the behavior and stamped the one existing document by hand so it matched.

And once that article was stamped, its 111 chunks flooded the "From this page" list as raw # Title\n\nSource: fragments and buried the four facts I had clipped. The fact tiers now exclude doc:% sources and a separate docs_by_page lists each document once.

None of these four are exotic. Each one is a reader or a writer that did not participate in the provenance contract the others followed. The tests were green because each test exercised the path that did participate.

The icon took four rejected drafts in one evening

The smaller build story is the toolbar icon, which turns green when the current page has memory from it. The first draft was a text badge; Chrome's fixed badge overlay ate most of a 16px icon. The second was a corner dot. The third was a pulse, which reads as an alert that never resolves. The fourth, the one that shipped, redraws the icon on an OffscreenCanvas from our own artwork, hue-shifting only the brand-blue pixels so the white eyes survive. A flat source-atop fill had painted over them.

The decision about which signal drives the color was harder than the drawing. Counting site-tier matches swallowed the icon on every page of a site. Counting title-only semantic hits dotted every Google results page. The icon goes green for exact facts and documents saved from this page, nothing broader. The probe result is cached for 30 seconds per page key, so a burst of tab switches is one round trip. That logic is in MCP-servers/Vodou-Console/src/vbb/page-probe.ts with five tests.

The invariant: readers of a governed column equal checks of its policy

Here is the failure class, stated so you can check it rather than nod at it.

For any column that a policy governs (source_host under a site mode, invalid_at under a soft delete), the set of code paths that read or write that column must equal the set of code paths that consult the policy. If there are N readers and N minus one policy checks, the policy is decorative on exactly one path, and that path is the one the user will hit.

The four defects in .74 all violate a sibling of this: two writers of memory_sources, one stamping provenance. Three readers of page memory, before P4, zero consulting a mode. The property is countable.

Run this against your own store before you trust your off switch

Take five minutes and count. First, find every place that reads the governed column, and every place that reads the policy. Substitute your own names.

# readers of the governed axis
grep -rn "source_host\|source_url" src/ --include='*.ts' --include='*.py' | grep -v test | wc -l
# places that consult the policy
grep -rn "siteMode\|site_mode\|isHostAllowed" src/ --include='*.ts' --include='*.py' | grep -v test | wc -l

Passing looks like the second number being close to the first, and a quick read of the diff between the two lists showing only helpers. Failing looks like 14 and 2, which is roughly what I would have seen before P4.

Second, check whether your writers agree on provenance:

SELECT source_kind, COUNT(*) AS rows,
       SUM(CASE WHEN source_url IS NULL OR source_url = '' THEN 1 ELSE 0 END) AS no_url
FROM memory_sources
GROUP BY source_kind;

Passing is a no_url of 0 for every kind that came from a page. Failing is one kind with rows and no URLs, which tells you which writer skipped the stamp. That query is how I would have caught add_text in seconds instead of on a live Wikipedia tab.

Third, if you soft delete, prove every reader filters:

UPDATE memories SET invalid_at = datetime('now') WHERE source_host = 'example.com';

Then hit each of your read endpoints for that host with curl and look for a nonzero count. Any endpoint that still returns those rows is a reader that did not get the memo. Undo with SET invalid_at = NULL on the same rows.

What the memory-pattern guides leave out

The published specs describe memory in tiers. The agent memory pattern spec names working, episodic, semantic and procedural, and its privacy rule is to redact PII at write time. The engineering playbook splits the same space into episodic, semantic and procedural. Both are right and neither says where the fact came from. A site mode is a rule about the source, not the tier, and write-time redaction cannot fire on a page the extension should never have read in the first place. The off mode has to sit in front of the read, which is a different enforcement point than the redactor.

Retraction is under-described too. The dependency-guided rollback paper makes the point that deleting a faulty memory leaves everything derived from it active. My forget --host has the same limit: it invalidates the source facts and does nothing about a fill plan or an answer that already used them. That is a real gap, not a footnote, and the paper is the clearest statement of it I have found.

Still open: a chunk that names its own page can undo a manual link

One limitation is live. mem page-link stamps a memory with a page by writing the column directly, and sync's COALESCE keeps that value on re-index. But a chunk whose own text carries a page: token reverts to that token's page on the next sync, overriding the manual link. Two writers of source_url again, with a precedence rule that favors the wrong one for this case. I noted it in the P2 commit and did not fix it in P4. If you count your writers with the query above, that is the kind of pair it will show you.

Source: Per-site memory off switches only work if every reader asks the same authority by Chad Priest, from Building Vodou in Public.

Your Agent's Memory Has No Idea Where It Was

Chad Priest — Wed, 26 Aug 2026 17:35:14 +0000

Every agent memory system I have read stores what was said and when. Almost none store where. The chunk knows its text, its embedding, a created_at, maybe a session id. It does not know that you learned it on a Wikipedia article, or typed it into a vendor onboarding form, or that it came out of a ChatGPT conversation you had on a specific page.

That missing axis costs you more than it looks like it should. Without it you cannot answer "what do I already know about the thing in front of me," you cannot forget one source without hunting chunk ids, and you cannot tell a user which of your stored facts came from their bank's website, which, the moment you have a browser extension, is the question that decides whether they trust you at all.

I spent two days adding that axis to Vodou's memory store. Here is what it actually took, including the four defects that every unit test in the repo said did not exist.

Three columns and a match endpoint that tiers by URL

Three columns on the memory chunk: source_url, source_host, source_ref. Then a match endpoint that, given a URL, returns what memory holds about it in four tiers.

// MCP-servers/Vodou-Console/src/api/page-match.ts — the tiers
// T1  exact page   facts whose source_url matches this URL
// T2  this site    facts from the same host
// T3  semantic     facts that match the page's title, not its URL
// T4  documents    Library docs saved from this page

On top of that: a browser extension that colors its own toolbar icon green when the current page has memory from it, a "Note about this page" box, insert-anywhere, typing suggestions, form fill from memory with a review card, and a per-site governance model that decides whether Vodou is allowed to look at the tab at all.

What happens when you switch tabs

The interesting design constraint is the last edge. The icon is green for T1 and T4 only. That was not the first thing I built.

Everything green, then four defects in build .74

Everything green, four defects

I finished the unit suites. page-match.test.ts, page-probe.test.ts, page-site-mode.test.ts: all passing, plus four new route tests for the note and link endpoints. Then I loaded build .74 as an unpacked extension with the DevTools Network tab open and used it like a person for twenty minutes: a real ChatGPT capture, a Wikipedia clip, a saved article, the keyboard shortcut.

Four defects. None of them visible to any test I had written.

A thirty-minute-old fact rendered as "18h old." The by-page query returned COALESCE(valid_at, created_at), and for a fact extracted out of a daily log, valid_at is the log day at local midnight. So a fact written at 12:30pm carried an instant of 00:00:00 and the age formatter did the arithmetic correctly on a value that was never meant to be read at that precision. The fix is not in the query, it is in the renderer: day-granular instants now render "today" / "yesterday" / "3d", never hours. A timestamp with a granularity is not a timestamp.

Two Wikipedia clips landed with an empty source_url. The right-click "Send selection to Vodou memory" handler sent the selected text and nothing else. The selection is the content, so the handler looked complete. It had no tab metadata, so both clips were stored with no page: findable by search, invisible forever from the page they came from. The whole feature, silently not working, on the exact path a user would try first.

The Library lane never stamped anything. There are two ways a page becomes a document: add_url and add_text. add_url wrote memory_sources.source_url and stamped its chunks. add_text did not. "Add this page to Vodou Library", the button an actual person clicks, routes through add_text. I had implemented the axis on the lane nobody uses. I mirrored it and hand-stamped doc #225 to match.

Then the fix broke the feature. Once that 111-chunk article was correctly stamped, it flooded "From this page." The panel filled with # Title\n\nSource: fragments, raw chunk headers, 111 of them, and buried the four facts I had actually clipped. Correct data, useless surface. Fact tiers now exclude doc:% entirely, and documents get their own list that shows each document once instead of each of its chunks.

Every one of these lived in the gap between "the function returns the right value" and "a person got what they came for." The tests asserted the first. Nothing asserted the second.

Four indicators in one evening, three rejected on sight

The icon was supposed to be the easy part. Show the user that this page has memory. I built four versions in one evening and threw away three.

A text badge with a count: Chrome's badge overlay is fixed-size and ate a 16px icon. Unreadable. A corner dot: too small to notice, and it does not say what it means. A pulse: motion in a browser toolbar reads as an unending alert. You notice it once and then it is a thing you are ignoring forever. The one that shipped is the icon itself drawn green, redrawn per tab on a transparent OffscreenCanvas from our own artwork.

That last one had its own trap. Hue-shifting the brand-blue pixels works. A flat source-atop fill does not: it paints over the white eyes in the logo and you get a green blob. MARK_MODE still keeps dot and pulse one line away, because I do not trust that I am done being wrong about this.

The harder question was not how to draw it but when. My first cut lit the icon for any tier hit. Two failures, both obvious the moment I used it:

Include the site tier (T2), and the count swallows the icon on any site you have ever stored anything from.
Include the title-only semantic tier (T3), and every Google results page lights up, because a results page's title contains your query, which is what you were searching your memory for.

Green means T1 and T4 only: facts and documents saved from this page. That is a claim narrow enough to always be true. An indicator that fires on a weak match is worse than no indicator, because a user calibrates on it once and then never looks again.

The permission model was the actual feature

The part I underestimated by the widest margin: this is a browser extension that reads the URL of every tab you open. That is not a feature detail. That is the whole trust question.

The gateway is the authority, not the extension. Site mode is off / suggest / collect. Banks, health portals, tax sites and sign-in hosts default to off: Vodou does not read the tab there until the user says otherwise. Resolution order is user rule, then sensitive default, then global default, and a rule covers subdomains.

Off is enforced in four places, not one: page-match returns empty without asking the daemon, page_probe returns no icon, and note and link both 403 unless the mode is collect. The extension-side typing suggestions ask the mode first and send nothing until it answers. Four tests on the resolver, four on enforcement.

Then mem forget --host, which is soft (sets invalid_at), covers subdomains, and counts Library documents rather than removing them. First panel click is a dry run that shows you the count; a second click within 8 seconds confirms. Verified live on w3schools.com: hide, 1, undo, 0.

The QA plan for all of this was two-phase and network-observable, which is the only kind of privacy test I now believe:

Phase A  page memory OFF  → switch tabs → assert ZERO page-match requests
Phase B  page memory ON   → accept disclosure → assert requests appear,
                             and the fact walnut-lantern-4417 shows under
                             "From this page" in the ChatGPT tab

Phase A is the important one. It does not test that the feature works. It tests that the feature is absent, from the network panel, where a skeptical user would look.

Test the privacy guarantee on the wire, not the return value

Your privacy guarantee is a claim about the wire, so test it on the wire.

Nearly every agent system I have seen tests its permission model by asserting the return value of a permission function. That is testing your intent. The user's actual question is "did you send my data anywhere," and the answer to that lives in the network panel, not in your unit suite. Write the test that asserts zero requests. It is a strange test to write, most test frameworks make asserting a non-event awkward, and it is the only one that answers the question that was asked.

Two corollaries, both of which cost me a defect each:

When you add a field, find every write path, not the one you were looking at. add_url and add_text both create documents. I stamped one. The one I missed was the one behind the button. Grep for every writer of the table, not every caller of the function you happened to open.

A stored timestamp has a granularity, and the granularity does not travel with it. A datetime column looks second-precise. A value derived from a log day is not. If you render it at a precision the data never had, you produce a confident lie: "18h ago" for something thirty minutes old. Carry the granularity, or render at the coarsest one you can guarantee.

Agent memory tiers say nothing about which page you were on

The public discussion of agent memory is almost entirely about tiers. The engineering playbook's agent-memory writeup splits long-term memory into episodic, semantic and procedural. Geodocs' pattern spec adds working memory on top and specifies consolidation, scoring, eviction and write-time PII redaction. Both are good. Both are about what kind of thing a memory is.

Neither has an axis for where the user was standing when it entered the store. That is not an oversight in the taxonomy so much as an artifact of the assumption underneath it: that memory arrives through your chat interface. Once memory can be captured from any page in a browser, provenance stops being metadata and becomes the primary key for two operations you cannot otherwise perform: "show me what you know about this" and "forget this source."

The Pockit production-memory guide is right that context windows are not memory and that persistence is the wall everyone hits. But durability cuts both ways, which is the point of "From Faulty Memories to Corrected Actions": persistence makes errors durable too, and deleting a bad memory leaves everything already derived from it in place. Their answer is dependency-guided rollback. My cheaper, dumber version of the same insight is that if every chunk carries its origin, "forget everything that came from this host" is one soft update instead of an archaeology project. Provenance is not just retrieval: it is the handle you need to undo.

Worth reading alongside these: agentmemory#648, a stability pass where two separate bugs turned out to share one root cause, a daemon wrapper surviving stop, and the dawnai memory-tooling commit, which adds a gated live-smoke suite for exactly the reason I am writing this post: some behaviors only appear against a real backend, so they gate a separate suite on a real key and keep it out of CI.

COALESCE lets a chunk's own page: token overwrite mem page-link

The stamping path has a known conflict. mem page-link writes the column directly, but re-indexing uses a COALESCE that prefers a page: token found in the chunk's own text. So a chunk whose text carries its own page token will revert to that token on the next sync, overriding the link. I know about it, it is in the plan, it is not fixed.

Form fill is two lanes and only one of them is deterministic. Learn-back, answers you accepted on this page before, or on this site, is exact and ordered, latest wins. Everything past that is one LLM call with a hard "only what memory supports, never invent" rule, which is a rule you write down and hope for. The review card exists because I do not trust that lane. Nothing is written to the page until you tick it, and it never submits the form.

Sensitive-host defaults are a list. Lists are wrong at the edges. A bank I have never heard of collects by default until somebody notices, and "somebody notices" is not a security model.

And the whole thing rests on one number I cannot measure from here: how many of those four live-only defects still exist in the paths I did not manually click for twenty minutes. Every unit suite in this repo is green. That was true before I opened the browser, too.

Source: Your Agent's Memory Has No Idea Where It Was by Chad Priest, from Building Vodou in Public.

Your retired button is still mounted, still handling clicks

Chad Priest — Wed, 26 Aug 2026 17:26:21 +0000

The button did nothing. No console error, no failed request, no red toast. I clicked it, and the page went on being a page.

I had spent the afternoon inside runInject(): the function that pulls a context block from the local gateway and drops it into whatever composer the host site is using. Edit, reload the extension, reload the tab, click. Nothing. On the third round I put a console.log on the first line of runInject. It never fired.

The button I was clicking was not wired to runInject. It was cbtn, the retired "🧠 My context" control from the previous design, and it called requestContext: the old in-page picker overlay. It had been superseded weeks earlier. It was still mounting on every page load, still painted with the same mark as its replacement, and still perfectly happy to handle a click.

Two entry points, one scope, no error

The shape that made it invisible:

// content.js — one IIFE, two mount paths
function mountContextButton() {
  if (window.__vodouContextButtonMounted === MOUNT_TOKEN) return;
  window.__vodouContextButtonMounted = MOUNT_TOKEN;

  function runInject(site, forceComposer, composer, onDone, manual, ctl) { /* current */ }

  function mountFab() {                       // the NEW control
    if (document.getElementById('vodou-fab-wrap')) return;
    // ...
    run(report) { runInject(site, true, findComposer(), undefined, true); }
  }
  mountFab();
}

// ...elsewhere in the same IIFE, outside that closure:
mountCbtn();     // the OLD control — cannot see runInject, calls requestContext

runInject lives inside mountContextButton()'s scope. Nothing outside that closure can call it. So the retired control could not have been calling the current path even if I'd rewired it, and that's the part that took me embarrassingly long: the scope was the proof, sitting right there, and I read it three times as a detail instead of as an answer.

The mount guards made it worse by working. Ours are versioned on purpose: MOUNT_TOKEN is the manifest version, so reloading the extension re-arms the guard in already-open tabs instead of hitting a === true and returning early. The new control's guard is keyed on its own id. The old control had its own guard, keyed on its own id. Both cleared. Both mounted. Neither knew the other existed.

The standard advice is about orphaning. This wasn't orphaning.

Every result you get searching for this is about context invalidation. The classic Stack Overflow thread on orphaned content scripts and samber/cc-skills' content-scripts reference both cover it well: after an update the old script keeps running, loses chrome.runtime, every message throws, so you guard with isExtensionAlive() and show a refresh banner. AuditBuffet's pattern names the visible symptom exactly right: "ghost UI that does nothing when clicked."

None of it covers my case. My extension context was fine. chrome.runtime.id was there. isExtensionAlive() returned true, messaging worked, and the ghost button was shipping in the current bundle. There is no invalidation event to hang cleanup off when the thing that retired your code path was a refactor.

The class: a retired entry point that still boots

This is not a Chrome bug. It's what happens whenever a new implementation is added beside an old one in overlapping scopes and the old mount is left in the boot sequence: React components whose replacement ships but whose old addEventListener still runs from a stale useEffect, embedded JS SDKs (Stripe/Intercom-shaped) where v1's mount call survives in the host's snippet next to v2's, any server-rendered page that emits two <script> tags for the same widget.

Retiring a code path means deleting its mount call, not deleting its callers: an entry point that still executes at boot is live code, and if its handler is in a different scope from your current implementation, it is by construction calling something else.

Here's the check, in your own console, on your own page. Chrome DevTools only, since getEventListeners is a console builtin:

for (const el of document.querySelectorAll('[id*="myprefix"], [class*="myprefix"]')) {
  console.log(el.id || el.className, getEventListeners(el));
}

Passing looks like one node per control, one click entry each. Failing looks like two nodes you only rendered once, or one node with two click listeners. Then expand the listener, click the source link, and read which file and line it lands on. If that line is inside code you believed you deleted, you just found it. Sixty seconds.

The backtick that eats the stylesheet

Second half of the same day, same file. The injected <style> is a 103-line template literal, content.js:1325-1428, with ten prose comments inside it explaining why each rule exists. 327 lines earlier there's a JS comment that reads // `manual` = a user-initiated trigger. Same writing habit, different container. A backtick inside that CSS comment terminates the template early, everything after it parses as JavaScript, and the file fails to parse, so nothing mounts at all, on all 22 hosts. You get Uncaught SyntaxError: Unexpected identifier pointing at a line nowhere near the backtick you typed.

The check is a shell loop, and it works on any repo that emits markup from template strings:

for f in $(git ls-files '*.js' '*.mjs'); do node --check "$f" || echo "BROKEN: $f"; done

Passing output is silence. Failing output is a filename and a line number that will not be where the mistake is: that's the tell, and it's why this takes an hour by eye and four seconds by parser.

Two rules out of one day. Grep your boot sequence for mount calls before you grep for handlers. And whatever delimiter your template literal uses, it is banned inside that literal, including in the comments, especially in the comments, because comments are where prose goes and prose is where backticks live.

Source: Your retired button is still mounted, still handling clicks by Chad Priest, from Building Vodou in Public.

Your isolated test lab isn't isolated: 3 rows, 8 daemons

Chad Priest — Wed, 26 Aug 2026 17:25:39 +0000

I wrote a script whose entire job is to break my system on purpose in a sandbox and print what every surface says. scripts/broken-lab.sh, 516 lines, committed as ca66f6c8. It found three real defects in its first hour, which felt great until I looked closer and found that three of the bugs it found were its own.

It wrote three rows into the live gateway.db. It health-checked a different process than the one it started. And it leaked eight daemons into a machine-wide process limit that the production stack shares.

The system under test was fine. The instrument was lying.

The harness wrote to production because the app resolved its own path

The gateway reads VODOU_PROJECT_PATH to decide where its SQLite file lives. What I did not know is that db.ts only trusts that variable if the directory already contains vodou-core.db, and it resolves the root once at module load. My lab set the env var, skipped the baseline step that seeds that file, and the gateway silently fell back to the repo root. Then it staged a graph fan and recorded three runs in the live database: the one file this script promises in its header never to touch.

Nothing errored. The harness passed. I deleted the rows by hand.

The fix is not "set the variable more carefully." The fix is that the harness stops believing it:

if port_is_taken; then
  # Isolation is asserted, never assumed. If the gateway created its DB
  # anywhere but the lab, every later reading describes the live system.
  if [ ! -f "$LAB/MCP-servers/Vodou-Console/gateway.db" ]; then
    say "ISOLATION FAILED — the lab gateway did not create $LAB/MCP-servers/Vodou-Console/gateway.db."
    say "  It is writing to the LIVE databases. Stopping before anything else runs."
    lab_gateway_kill
    return 1
  fi
  return 0
fi

A 200 on your port is not proof that it's your process

Second bug, and this one cost a full debugging cycle. Startup waited for /api/health to return 200 and called that success. But a leftover gateway from an earlier lab run still held :8791, so my new process logged Refusing to start a second instance, exited, and the health probe cheerfully passed: against the stranger. Every subsequent request hit another lab's database. The new lab's gateway.db was never created. Then the kill test SIGKILLed a pid that had already died and reported a clean recovery.

Now the loop checks kill -0 "$pid" on the pid it spawned before it believes any 200, and aborts outright if the port is taken before boot.

The third bug: the lab gateway spawns its daemon and worker detached, from the repo binary, so kill -9 -PGID misses them and pkill -f "$LAB/vodou-core" can never match. The lab's path isn't in their argv. Eight orphans accumulated in one afternoon, and since VODOU_MAX_PROCESSES counts machine-wide, the next run was refused with 6 vodou-core processes are already running (limit 5). The harness had started starving the machine it promises not to touch. The only place the lab's identity survives is the environment, so that's what the reaper matches now: ps -E -o pid=,command= | grep -F "VODOU_PROJECT_PATH=$LAB".

Any harness that discovers its dependencies like production will grade production

The class: any harness that discovers its dependencies the same way production does will eventually grade production. Not "uses the same database". discovers the same way. Config file, env var with a fallback, service discovery, a well-known port, a shared process pool. It shows up as pytest fixtures against the shared dev Postgres, LangChain eval suites pointed at the production vector store by an unset INDEX_NAME, Compose suites reusing host ports, and any MCP or daemon test that spawns subprocesses with no cleanup barrier.

Testcontainers: Your Isolated Tests Are Still Lying To You is right that a shared dev database is a solved problem and half-isolation is the real trap. What it misses is my case: a throwaway container guarantees a clean database exists, not that the app under test used it. My isolation was correct in the harness and discarded by the callee. Closer is Your test suite is lying to you about which process it's in, where a correct guard ran in the wrong process. Same family. My version is worse in one way: I didn't have a guard that ran in the wrong place, I had a probe that couldn't tell processes apart at all.

A harness must assert on every run that the process answering it is the pid it spawned, and that the writes it made landed under the path it created. Both are resolved by the system under test, not by the harness. That's checkable: grep your test setup for a readiness probe with no pid comparison, and for a teardown that trusts argv.

Five-minute check against your own stack, no Vodou anything:

# 1. Does your suite touch production? Fingerprint before and after.
psql "$PROD_URL" -tAc \
  "select relname, n_live_tup from pg_stat_user_tables order by 1" > /tmp/before
pytest -q            # or npm test, go test ./...
psql "$PROD_URL" -tAc \
  "select relname, n_live_tup from pg_stat_user_tables order by 1" > /tmp/after
diff /tmp/before /tmp/after && echo ISOLATED || echo "YOUR SUITE WRITES TO PROD"

# 2. Is the thing answering your health check the thing you started?
./run-test-server & echo $! > /tmp/mine.pid
curl -sf localhost:8080/health >/dev/null && echo "health says 200"
lsof -ti tcp:8080 -sTCP:LISTEN > /tmp/listening
grep -qx "$(cat /tmp/mine.pid)" /tmp/listening \
  && echo "same process" || echo "TESTING A STRANGER: $(cat /tmp/listening)"

# 3. Do you leak? Count before and after.
pgrep -fc myservice   # run this before the suite, and again after

Passing looks like: empty diff, "same process", identical counts. Failing looks like a table that grew by three rows nobody can explain, a pid in /tmp/listening that isn't yours, and a count that climbs one per run until something unrelated starts getting refused. (n_live_tup is an estimate; for small tables use exact count(*) per table.)

The rule I'd give a stranger: a test that cannot prove which process it talked to and which file it wrote has not produced a result, it has produced a sentence. Make isolation an assertion with an abort, not a configuration you set once and trust.

Source: Your isolated test lab isn't isolated: 3 rows, 8 daemons by Chad Priest, from Building Vodou in Public.

Why My LLM Agent Fabricated Numbers From Stale Context

Chad Priest — Wed, 26 Aug 2026 17:19:58 +0000

I asked my own system what my CPU was doing. It told me, confidently, with a number. The number was ten minutes old. Nothing had measured anything on that turn.

Then it did it again in a different shape: it produced a timing table for MCP tool calls that it had reasoned out of the source code, and a set of memory statistics carried forward from an earlier run where get_memory_info had actually executed. Both were presented as measurements. Neither was measured.

That is the worst class of bug in an agent product. A crash is honest. This was the system lying to me in my own voice, about my own machine, using data it had legitimately collected at some point in the past.

The symptom: replayed tool output presented as a live measurement

Vodou's gateway (MCP-servers/Vodou-Console/) assembles context for every turn: memory recall from a SQLite store, plus tool output when a query routes to an MCP tool. Tool output goes to the model inside an <active_context> block.

There is a deliberate replay path. If you ask "what's my CPU at," get an answer, and then say "huh, that seems high," the second message doesn't route to a tool. It's conversational. So the gateway re-injects the previous turn's tool output so the model can talk about what it just said. Reasonable feature. Here it is, llm.ts:3195:

// BrainLoader skipped (conversational message) — still re-inject stored context if available
const stored = _lastOiContext.get(conversationId);
if (stored && Date.now() - stored.timestamp < 600_000) {
  oiResults = stored.oiResults;
  stored.timestamp = Date.now();
  console.error(`[BrainLoader] skipped (conversational) — reusing stored context (${oiResults.length} chars)`);
}

If you grep your own logs for reusing stored context, that line is where I started.

Three days blaming the TTL

Look at line four of that block. stored.timestamp = Date.now(): the sibling site at llm.ts:3099 even labels it // refresh TTL on use. Touch the cache and its age resets. A ten-minute window that refreshes on every read is not a ten-minute window; in a long conversation it never expires. Stale context reads as fresh forever.

I was sure that was it. I wrote up a fix: stop refreshing on read, and stamp the replayed block so the model knows what it's holding.

<active_context fresh="false" age_s="412">

Clean. Honest. The model gets provenance, decides for itself, and the whole thing is about fifteen lines.

I was wrong for three days, and the tell was sitting in the transcript the whole time: on the turn where Vodou reported the stale CPU number, the daemon had run the tool. Fresh output existed. The unbounded TTL explains how stale data survives. It does not explain how stale data beats fresh data that is present in the same request.

I also chased a second wrong theory in parallel: that this was ordinary model hallucination and the answer was a stronger anti-hallucination anchor in the system prompt. That turned out to be a real finding but a different bug. Three of the five provider functions get no ground-truth block at all, so BYOK OpenAI and Ollama users are running a materially different product than the managed tier and nothing in the UI says so. Worth fixing. Not this.

The measurement that cracked it: count the writers, not the readers

I stopped reading the consumers and enumerated the producers. Every code path in the entire system that can write text into a model request. One row per site, file and line.

The plan I was working from budgeted for five. There were seven.

Number six was the one that mattered. The Rust daemon has its own tool-output channel that I had forgotten about, and it does not write into <active_context> at all. It writes a ### Vodou Tool Results (auto-routed) heading into the memory lane, and it has a second cached variant that does the same. Two independent producers, two different lanes, one buffer at the far end.

(Number seven was the SessionStart hook's workspace bootstrap: AGENTS.md plus MEMORY.md, 25,293 bytes, consistent across sampled sessions, counted by nothing.)

Once both writers were on the same page, the bug was obvious.

Root cause: a guard that tested truthiness instead of provenance

Five sites in llm.ts, 5437, 5994, 6200, 7099, 7609, one per provider family, contain this:

// Fix 2: strip tool results block from system prompt — it's already in <active_context> via oiResults
const memoryForSystem = (oiResults && memoryContext)
  ? memoryContext.replace(/### Vodou Tool Results[\s\S]+/, '').trim()
  : memoryContext;

The intent is deduplication. If tool output is already going to the model in <active_context>, don't also ship the daemon's copy in the system prompt.

The guard is oiResults &&. It asks whether cached tool output exists. It never asks whether that output came from this turn.

So: conversational message, BrainLoader skipped, replay fills oiResults from ten minutes ago. Meanwhile the daemon auto-routes the query and puts genuinely fresh tool output into memoryContext. The strip fires, because oiResults is a non-empty string. [\s\S]+ eats the fresh block and everything after it. The model receives the ten-minute-old reading and nothing else.

The comment on that line asserts the deleted content is safely present elsewhere. It isn't. A different, older thing is. The code was working exactly as commented and the comment was describing a state that didn't exist.

That's the whole bug. Fresh output deleted to make room for stale output, five times, in a line whose comment claims the opposite.

The obvious fix is wrong on the OpenAI-compat path

My first instinct was to concatenate the fresh block back onto the system prompt when the guard rejects the replay. On three of the five provider paths, that works.

On chatWithOpenAICompat it silently breaks something else, and this is the part I'd have shipped if I hadn't gone reading. That function has a stable-prefix mode where the system prompt is deliberately frozen:

const STABLE_PREFIX = process.env.VODOU_COMPAT_STABLE_PREFIX != null
  ? process.env.VODOU_COMPAT_STABLE_PREFIX === '1'
  : currentProvider === 'vodou';
// ...
if (STABLE_PREFIX) {
  systemPrompt = staticParts;                  // frozen → cacheable prefix
  lateContextBlock = memoryForSystem || '';    // volatile → relocated to a late turn
}

The system prompt is staticParts and nothing else, on purpose, because byte-identical request fronts are what make the provider's prompt cache hit: 87–97% on warm turns with a roughly two-turn warm-up. Anything query-dependent gets relocated into the message array instead, spliced in as a late system turn just before the current user turn:

if (STABLE_PREFIX && lateContextBlock) {
  const insertAt = Math.max(1, m.length - 1); // before the trailing current-user turn
  m.splice(insertAt, 0, { role: 'system', content: '### Relevant context for this turn\n\n' + lateContextBlock });
}

Append fresh per-turn context to systemPrompt on that path and you don't lose the context. You lose the cache, on the tier where cost actually matters, and the only symptom is a bill. systemPromptStaticPrefix() also has two arities across the provider families, which is the second way that "just concatenate it" patch fails to compile in one place and compile-but-misbehave in another.

That is why the repair isn't merged yet. Five nearly-identical strip sites across five provider functions is roughly 27 edit points for the full set of changes, and I'm not hand-editing 27 sites in a file this hot. The single context-assembly seam gets built first; the strip repair, the replay deletion, and a regression test land in it.

The regression test is the part I'd write first if I were you: on a turn where the daemon auto-routes but the gateway's own router does not match, assert that the fresh ### Vodou Tool Results block reaches the model. That assertion fails today.

The policy change that came out of it is one line: replay instructions, never replay measurements. Skill text, menus, operating rules. Safe to re-inject, they don't decay. A CPU reading, a row count, a disk figure: those are true at an instant and lies forever after. That distinction also killed 341 entries in my conflicts queue that turned out to be CPU readings from different runs disagreeing with each other, which is not a contradiction, just two clocks.

Here's the rule I'd hand to a stranger. If two code paths can write to the same buffer, every dedupe guard between them must key on provenance, not existence. if (cached) is not a freshness check. if (cached.turnId === currentTurnId) is. And before you tune any of the readers, enumerate the writers with file and line numbers, because the number is higher than your architecture diagram says. Mine said five.

Source: Why My LLM Agent Fabricated Numbers From Stale Context by Chad Priest, from Building Vodou in Public.

Every copy of the rule agreed. That was the bug.

Chad Priest — Wed, 26 Aug 2026 17:19:17 +0000

Any AI system past a certain size has the same rule written down in more than one place. A slug format. A status word. A receipt string. A retry policy. You wrote it twice because the two surfaces are in different languages, or different processes, or one of them ships to a browser and the other doesn't. You left a comment saying MUST match.

Both copies agree today. That is not a contract. That is a coincidence you have not audited yet.

I spent three days in August auditing every one of those coincidences in Vodou. Thirty-two commits, eighty-seven files. Almost none of it was new features. The interesting part is what the audit found, and how much of it no test could have caught.

A finding is a number in FINDINGS.md, and walks are immutable

The unit of work is a finding. Someone walks a flow end to end as a user would, writes down everything that contradicts something else, and the contradictions land in a FINDINGS.md with a number. Walks are immutable once written. Fixes cite the number in the commit subject: fix(coherence): F13 — the @doc: token is minted once, by the code that resolves it.

There is no coherence module. No vodou-core coherence command. No scheduled job. It is a documents-and-commits discipline, and the only executable parts are at the two ends: a lab that breaks the product on purpose, and a commit guard that refuses the specific defects the audit proved were real.

What happens to a finding

The slug rule was written four times, the fifth copy was the test

The slug rule was written four times, and the fifth copy was the test.

Vodou resolves @doc:contract-2026 tokens to attached documents. The slug rule, how a filename becomes a token, lived in the resolver, in the library page, and twice inline in the browser panel, once for the documents lane and once for the page-memory lane. I had expected two copies. There were four.

The fifth was in the end-to-end test, in a comment that read verbatim from sidepanel.js. That test could only ever prove that two copies of one expression agree with each other. It had passed every day of its life and was structurally incapable of failing for the reason it existed.

All four product copies agreed. That is the whole problem. The day one drifts, the failure is silent and total: a token minted one way and resolved another names a document that does not exist, so the system tells you it attached your contract and then answers about something else.

Synchronising the copies would have been the wrong fix. I deleted the deciders instead. /api/library, /api/library/match and /api/page-match now mint the slug with the resolver's own function and ship it on the row. The surfaces paste what they were handed. Where a client needs a fallback, it falls back to the id, which the resolver already accepts.

A test that goes red because the code improved is measuring the wrong thing.

The same week I found three byte-identical copies of the phrase 4 memories · 2 tools · 1 skill: one in the content script injected into ChatGPT, one in the panel, one in Console Two. Each built the counting and the pluralisation itself.

There was already a parity test for this. It grepped the panel's render function for three ternaries. When I moved those rules into a shared receipt.js, the test went red, not because the receipt was wrong, but because the ternaries had moved somewhere better. I had written a test that asserted the location of a bug's workaround. It had to be replaced, not repaired.

Independent implementations of one rule

The lab I built to test broken states was not isolated.

Phase 0 was scripts/broken-lab.sh: induce daemon-down, empty-account, unreadable-db, no-memory in a throwaway instance, then print what every surface says in each, verbatim and side by side. It found three real defects in the first hour. The best one: a status command was reporting the health of a different installation entirely, so it answered "healthy" for a stack that was face down.

The first version of the lab also wrote to the live database, health-checked a stranger, and leaked eight daemons. A harness that claims isolation has to prove it on every run, not in its README. It now owns its project root, its DBs and its socket, calls no start or stop script, and kills no pid it did not spawn.

One word meant five things, and two of my renames were wrong.

The vocabulary pass was the one I thought would be mechanical. It was not. Two of the renames changed a word that was correct in its context, and I had to back them out. A separate finding, F43, turned out to be false: the contract I claimed was missing already existed, and what had actually failed was the guard's ability to see it. Another was already fixed months earlier, and the only real defect left was a report implying otherwise.

Roughly one finding in six did not survive contact with the code. If your audit has a 100% confirmation rate, you are writing down conclusions, not observations.

Two questions rode one exit code.

runtime-status printed Overall: down and exited 0. mem search exited 1 for the same condition. So vodou-core runtime-status && start_work walked straight into a dead stack.

They disagreed because the exit code was carrying two different questions, did the command work and is the subject healthy, which are not the same answer for a reporting command. They now get separate codes, the way systemctl has always done it.

# 0 = healthy   2 = reported, and unhealthy   1 = the command itself failed
vodou-core runtime-status; case $? in
  0) start_work ;;
  2) echo "stack is down; the report is trustworthy" ;;
  1) echo "could not determine anything" ;;
esac

An unknown verdict is never healthy. Finding the two existing callers and teaching them this was half the work: both read any non-zero as "the probe is broken", so one printed a fallback one-liner underneath a complete report, and the other discarded the payload and said "unavailable" for a stack that had just described itself in detail.

The version string was never going to answer "which build is this".

We once lost 38 hours to a daemon quietly running the previous build. Both sides reported the same version. Three of our extension folders still do. What actually discriminates a build is the file it was loaded from, so that is what every long-lived process now reports: path, size, mtime, and whether debug assertions are compiled in. One command cross-checks them.

The ordering is load-bearing and easy to get backwards. The binary swap does rm then cp, so an identity read lazily at status time would describe the replacement and report a stale process as current. It has to be stamped at process start or it is worse than nothing.

More than one implementation is an outage with a delay fuse

A rule with more than one implementation is an outage with a delay fuse. The copies agree right up until someone improves one of them, and improvement is exactly what your review process rewards. So the fix is never "sync the copies". It is: delete every decider but one, and have the others paste what they were handed.

Where you genuinely cannot share code, different language, different runtime, ships to a different place, you do not get a contract, so buy the next best thing: a test that reads both implementations from their real locations and asserts they produce the same output for the same input. Not one that greps one of them.

And check what your parity tests actually compare. If a test's fixture was copy-pasted from the implementation, it is a mirror, not a check.

Four slug copies lived under a comment saying MUST match

The instinct is to solve this with review prose: a CONVENTIONS.md, a comment reading MUST match, a checklist item. We had all three. The four slug copies were written under a comment saying they must match.

The working idiom in this repo is commit guards, and the audit's mechanical rules became one: scripts/coherence-guard.py, with scripts/test-coherence-guard.sh covering eleven hermetic cases. It reads staged added lines on shipping surfaces only, and blocks three things the audit proved were real: an internal enum reaching textContent untranslated; a *_count that is displayed but that nothing ever writes; an iframe framing one of our own surfaces with no declared min-width. When a flag is deliberate you say so in the diff, in a comment the guard reads.

The second half is measurement, because some rules are not statically checkable. Grading has to compare two records of one event: capture-to-searchable latency per lane, a receipt's claimed count against the ids it actually recorded, last_run_at against the newest run row. A grader with no evidence answers unknown, never ok. That distinction is the whole point. "Nobody looked" and "fine" are different states, and only one of them is safe to ship on.

Console Two cannot import the shared receipt module

Console Two is a separate codebase and cannot import the shared receipt module, so that phrase still has two homes and a parity test standing between them.

The guard only sees staged added lines on four directory trees. Move a violation into an unlisted path and it passes.

Committed source and the committed binary in this repo drift apart. A fix can be at HEAD and still not be what the machine runs. Build identity makes that visible now; it does not make it stop.

And one measurement I would rather not have: an extraction rule nobody had ever checked is holding at 30%. It is written down, numbered, and open.

Source: Every copy of the rule agreed. That was the bug. by Chad Priest, from Building Vodou in Public.

SQLite FTS5 quietly ANDs your search terms

Chad Priest — Wed, 26 Aug 2026 17:13:35 +0000

Someone asked my search box a normal human question and got back {"results":[]}.

Not an error. Not a timeout. An empty array, in 8ms, with a 200. The corpus it
searched has 52,058 chunks in it and definitely contains the answer. I know it
does, because a two-word version of the same question returned twenty hits.

I spent the first twenty minutes assuming this was a ranking problem, because
that's the interesting kind of problem and I wanted to have it. It was not a
ranking problem. It was one space character.

The symptom: results decay to zero as the query gets longer

Here's the thing that made it obvious, once I stopped theorizing and just ran
the same query at increasing lengths:

Query	Hits
`daemon`	1,521
`daemon socket`	162
`daemon socket memory`	153
`daemon socket memory search`	27
`daemon socket memory search Tuesday`	0

Monotonic decay. Every word you add can only ever remove results. That is not
what relevance looks like: relevance is noisy, it goes up and down. A clean
staircase to zero is a set intersection, and set intersections come from
AND.

The cause: space-joined tokens are an implicit AND

The query builder was doing something that reads as completely reasonable:

const tokens = q.split(/\s+/)
  .map(t => t.replace(/[^\w'-]/g, ''))
  .filter(Boolean)
  .map(t => `"${t}"`);

// ...
WHERE memory_fts MATCH ?   // tokens.join(' ')

Split the query, strip punctuation so it can't blow up the parser, quote each
token so a stray OR or NEAR in user text is treated as a literal, join it
back together. Four defensive decisions, all individually correct.

The problem is the last one. In FTS5, whitespace between two terms is the
implicit AND operator. "daemon" "socket" doesn't mean "look for these two
words." It means "return only rows containing both." That's documented FTS5
behavior and it's the
right default for a query language. It is an absolutely terrible default for a
search box, because a search box receives sentences.

Do the math on a real query. Someone types:

I was thinking about the deploy pipeline yesterday and also need to remember
to renew the domain on Tuesday, plus what was that socket timeout we hit, and
can you check the graph frontend branch

That's 33 tokens. For that to return anything, one single document has to
contain all 33 of those words. deploy and domain and socket and
branch, in one chunk. No such document exists in any corpus, which means this
query returns zero results, and so does every other query of that shape. The
failure rate isn't "sometimes": it's 100% of prose queries, forever, with
no error message.

The reason it survived so long is that it works beautifully for the case you
test it on. You type daemon and get results. You type dogs name and get
one stray hit, and you assume the data is missing.

The second bug hiding under the first: `dogs` is not `dog's`

While measuring the above, I hit a separate cliff:

dog          → 94 hits
dog's name   → 23 hits
dogs name    →  1 hit

The sanitizer keeps apostrophes ([^\w'-]), which is deliberate: you want
don't to survive as one token. But that means dogs and dog's are two
different tokens, and only one of them is meaningfully in the index. Nobody types the
apostrophe. Every real user query is the broken spelling.

Two independent near-zero paths, both invisible, both looking exactly like
"we never stored that."

The debugging lesson: measure the shape, not the instance

The thing that actually cracked this wasn't reading code. I read that code twice
and it looked fine both times, because each line is fine.

What cracked it was running the query five times at five lengths and looking at
the shape of the curve. A ranking bug produces noise. A tokenizer bug produces a
cliff. An AND bug produces a staircase. You can identify the class of failure
from the shape before you know anything about the cause, and that tells you
which file to open.

I'd generalize it like this: when a query returns nothing, don't debug the
query: debug the family it belongs to. Vary one dimension (length, casing,
punctuation, term rarity) and plot it. Five curl calls got me to the right line
faster than an hour of reading would have.

The fix: tokens.join(' OR ') alone trades zero hits for garbage

The obvious patch is tokens.join(' OR '). Don't ship only that: you'll trade
zero results for garbage, because now a stopword like the pulls in half the
corpus and the top hit is whatever document says "the" the most.

What you want is a tiered fallback:

// 1. Try AND — if the user typed 2-3 precise terms, this is the best answer.
let rows = search(tokens.join(' '));

// 2. Zero hits? Fall back to OR, but cap tokens and drop stopwords first,
//    so ranking has something to work with instead of everything.
if (rows.length === 0) {
  const terms = tokens.filter(t => !STOP.has(t)).slice(0, 8);
  rows = search(terms.join(' OR '));
}

AND-first preserves precision for the short queries where precision is what you
want. OR-fallback means a sentence degrades into worse ranking instead of
no results. Those two failure modes feel completely different to a user: one
is a search engine having an off day, the other is a broken product.

And normalize possessives at index and query time both, or you will keep
shipping the dogs bug in new places.

Zero results as the query grows means intersection, not search

If a search interface returns zero results more often as the input gets
longer, it is doing set intersection and calling it search.

Any search box wired directly to MATCH has this bug. It will pass every test
you write, because you write tests with keywords, and it will fail every query
a human types, because humans type sentences. Go paste a full sentence into your
own search box right now. If you get an empty state, you know what to look for.

Source: SQLite FTS5 quietly ANDs your search terms by Chad Priest, from Building Vodou in Public.

A silent failure looks exactly like a feature you never built

Chad Priest — Wed, 26 Aug 2026 17:12:54 +0000

Your system has memory. Your user is typing into someone else's chat box.

That gap is the whole problem, and it is not a retrieval problem. The place your context needs to arrive is a page you do not control, rendered by a company that has no interest in your product, whose DOM changes on their schedule. Every design you can reach for there is bad in a different way. A browser popup closes the instant the user clicks back into the page, which is the exact moment they need it. In-page injection means you are a permanent guest, mounting your UI into a tree the host owns and re-mounting it every time the host re-renders.

We ran the second option for months. The memory picker was injected in-page, with a 3-second remount timer, on all 22 chat hosts we support. It worked. It also meant that every user with the extension installed had a timer firing four times a minute on every AI site they visited, forever, to fight a re-render that usually never came.

Between 2026-07-29 and 2026-08-02 I moved the whole surface into Chrome's side panel and retired the popup. 45 commits, 43 files. Most of them were not the feature.

The popup is gone: one panel holds Memory, Activity, Settings

The toolbar icon now opens a side panel instead of a popup. The panel holds three views: Memory, Activity, Settings. The memory picker lives there, so the 22-host timer is gone. Pairing, connection state, gateway config and import all moved in, so there is one surface instead of a popup that could only be six inches tall and an options page nobody found.

The panel persists while you type. That is the entire reason it exists.

The content script still has to exist, because inserting text into the composer and reading the reply out of it can only happen in the page. But it no longer owns any UI worth keeping alive.

Where the surface sits

The spike: can chrome.sidePanel.open() fire from a commands handler

The spike existed to make an unknown observable. Before designing anything around the side panel, I needed one fact: does chrome.sidePanel.open() work from a commands keyboard handler, or does Chrome insist on a click? I wired both paths in a throwaway build and made both fail loudly, because a silent failure there is indistinguishable from "the shortcut isn't bound," and we would have re-derived the same question the next session. That instinct was right, and it is the only reason the next bug took an hour instead of a day.

The gesture does not survive an await. sidePanel.open() may only be called in response to a user action, and Chrome's user-gesture flag is consumed the moment you yield. My first version awaited setOptions() and then called open(). It threw every single time. There were two awaits in the way, not one: the command listener was async and fell back to await chrome.tabs.query(...) when it had no tab.

// wrong: the gesture is spent before open() runs
async function openPanel(tabId) {
  await chrome.sidePanel.setOptions({ tabId, path: 'sidepanel.html' });
  await chrome.sidePanel.open({ tabId });   // "may only be called in response to a user action"
}

// right: privileged call first, bookkeeping after
function openPanel(tabId) {
  chrome.sidePanel.open({ tabId });                 // synchronous entry, first async thing
  chrome.sidePanel.setOptions({ tabId, path: 'sidepanel.html' })
    .catch(err => console.error('[vodou] setOptions', err));
}

The command listener is now synchronous and uses the tab the event hands it. If a tab is ever missing it logs loudly instead of awaiting a query, because the await is precisely what breaks it.

A shortcut can be bound to nothing and look bound. I shipped "mac": "Ctrl+Shift+M". Chrome converts a bare Ctrl in a mac suggested_key into Command. So the toggle became Cmd+Shift+M, which is Chrome's own profile switcher and cannot be overridden. Nothing was bound to literal Control+Shift+M, so pressing it did exactly nothing. No error, no log line, no entry anywhere a user could see. The second one was worse: inject-context became Cmd+B, which is bold on all 22 sites, and content.js had carried e.ctrlKey && !e.metaKey for months with the comment "Cmd+B stays the site's bold on macOS." I shipped a manifest that contradicted my own code's comment. Had Chrome actually bound it, Ctrl+B users would have kept working while bold quietly stopped functioning everywhere.

{ "commands": { "toggle-side-panel": {
    "suggested_key": { "default": "Ctrl+Shift+M", "mac": "MacCtrl+Shift+M" } } } }

MacCtrl is the literal Control key, which is what the content-script gate tests for. A test now rejects a bare Ctrl in a mac key.

Three builds claimed one version line. We ship a store build, a sideload build, and a sideload-only build. Each got bumped by whatever work happened to touch it, so they had drifted to .10, .9 and .8. The spike landed in sideload at .9 while the store build sat at .10 without the side panel, which means the higher number was the older code. That is backwards for the one thing a version is for. Worse, dist/ zips are named by version, and a 0.5.97.9-store.zip already existed from a different package. One version string naming two artifacts is how you upload the wrong zip to Google. The fix was not a numbering convention. It was porting the panel into all three builds so that equal versions mean equal features, then setting all three to 0.5.97.11.

Then the stupid ones, which cost the most wall-clock. A backtick inside a CSS comment killed the entire content script, because the stylesheet lived in a template literal. The resting state of our mark rendered as a 42x40 oval, twice, because BIMI's viewBox cropped it. content.js kept throwing after every extension reload, because it was still evaluated in a dead extension context. We sent on a socket that had closed during a chrome.tabs.query. One out-of-order const removed every in-page button on every site.

The audit found what the feature work hid. Before submission I read the diff cold and found two leaks, a keystroke hijack and around 450 dead lines left over from the popup era. The copy said "your chats never leave your computer," which was false as written, so it changed. And in the same sweep I discovered I had deleted the activity log's only write. The Activity view looked completely healthy the whole time, because it was rendering rows written before the deletion.

Capture across 22 sites is a fixture farm. One parameterized extractor covers 20 of them, with per-site save selectors verified one at a time. A Kimi reply was stored as fragments while the console reported success. Auto-attach at send missed Perplexity's send button, then Kimi's, then Kimi's disabled state, then needed a per-site override for Manus, and finally a guarded positional fallback for buttons with no stable marker. NotebookLM shipped with an invalid Chrome match pattern. OpenRouter needed incremental scroll-and-collect before it worked at all. That got us to 20 of 22.

The lesson: a silent failure is indistinguishable from a missing feature

Every expensive bug on this list has one shape. Something did not happen, and nothing said so.

The shortcut fired into an unbound key. The panel refused to open with an error only the console saw. The activity feed rendered stale rows after its writer was deleted. Kimi's capture reported success and stored splinters. The version number said the store build had the newer code.

This is the dominant failure class in agent systems, not just browser extensions. A tool call that returns 200 and writes nothing. A retrieval that returns an empty list because the index is empty, which is byte-identical to a retrieval that correctly found no matches. A guard that fails open on timeout. In all of them the system's own report is consistent, and the only way to know is to compare two independent records of the same event.

Three things you can do today, on any stack:

Assert the effect, not the call. After a tool run, read back the row you claim to have written and compare it to what you sent. We now grade capture by comparing a receipt's count against the ids it actually recorded, because a count is written by the same code path that is lying to you.

Make absence a distinct state from failure. unknown is a valid answer and it is not ok. A health check with no evidence should report that it had no evidence.

Spend a half-day making a gate observable before you build on it. That is what the spike was. It cost four hours and it is the reason the gesture bug was diagnosable from my own code rather than from a user report three weeks later.

The narrower lesson, which generalizes further than it looks: authority is time-scoped. Chrome grants the gesture for the synchronous duration of your handler and await spends it. The same shape shows up in approval windows, short-lived tokens and transaction contexts. Do the privileged thing first. Do the bookkeeping after, with its own error handler.

Building Effective AI Agents is about the loop, not the browser surface

The good guidance on agent design is about the loop. Anthropic's Building Effective AI Agents is right that the successful implementations use simple composable patterns rather than frameworks, and OpenAI's practical guide to building agents is right about orchestration and guardrails. Neither has much to say about the surface, and the surface is where an agent that lives alongside a user actually fails.

Two projects are working the same seam. The claude_codex_bridge sidebar provider activity plan states the rule I arrived at by breaking it: the sidebar remains a client of one authority and must not become a separate authority for identity or state. Our panel reads storage and messages the worker; the worker owns the socket. When I let three build artifacts each own their own version number, I got the thing that plan is written to prevent.

And Cordum's ADR-010 contains the cleanest statement of the failure class I know, about a Claude Code PreToolUse deny hook: HTTP hooks can deny with a 2xx JSON response, but connection failures, non-2xx responses and timeouts are non-blocking. A guard whose failure mode is "allow" is not a guard. That is my unbound Mac shortcut with higher stakes.

For the vocabulary of the problem, Agent Surface is worth reading. Making software legible to agents is the mirror image of what we did here, which was making an agent legible inside software that has never heard of it.

Two of 22 sites still have no verified save path

Two of 22 sites have no verified save path. I know which two.

The positional fallback for unmarked send buttons is a guess with a guard on it. It will break, and when it breaks the symptom will be an auto-attach that silently does nothing, which is the exact failure class this post is about. It is the piece I trust least.

Per-site selectors are fixtures against DOMs owned by other people. There is no version of this that stops needing maintenance. We test them, which turns a break into a red suite instead of a support ticket, and that is the whole of the mitigation.

Three build folders still exist with three copies of sidepanel.js. One version line makes them honest. It does not make them one file.

The keyboard shortcut is discoverable now, because it is a real commands entry that appears in chrome://extensions/shortcuts and can be rebound. For its first several months it was a keydown listener in the content script, which meant it was invisible, unbindable, and dead on any host where injection failed. Nobody reported it. That is the point.

Source: A silent failure looks exactly like a feature you never built by Chad Priest, from Building Vodou in Public.

Node execFile was 2.7x slower: the spawn cost nothing

Chad Priest — Wed, 26 Aug 2026 17:06:32 +0000

Same machine, same daemon, same question. The gateway got its memory context in 1495ms. The MCP server got it in 4041ms. Both of them are code I wrote, in the same repo, for the same operation.

Unix socket in llm.ts, execFile in vodou-memory/index.js:38

The gateway talks to the Rust daemon over a Unix socket, one round trip, in MCP-servers/Vodou-Console/src/llm.ts:

const request = JSON.stringify({ cmd: 'prompt', payload: { hook_json: ... } }) + '\n';
const client = net.createConnection({ path: sockConnectTarget(sockPath) }, () => {
  client.write(request);
  client.end();
});

The MCP server shells out to the CLI instead, in MCP-servers/vodou-memory/index.js:38:

function core(args, timeoutMs = 20000) {
  return new Promise((resolve, reject) => {
    execFile(VODOU_CORE, args, { cwd: PROJECT_ROOT, timeout: timeoutMs, maxBuffer: 4 * 1024 * 1024 }, ...);
  });
}

VODOU_CORE is a 33MB Rust binary. So my diagnosis wrote itself: fork, exec, page in 33 megabytes, parse argv, tear it down. Of course that's slower. Delete the second implementation, point it at the socket, ship.

Forking the 33MB binary measured zero milliseconds, twice

Before deleting anything I measured the thing I was about to blame:

$ /usr/bin/time -p ./vodou-core --help
real 0.00

Zero. Twice. Fork/exec of a 33MB binary on an M-series Mac is free at this resolution, and I'd have known that a year ago if I'd ever checked instead of assuming.

The real answer was already sitting in the JSON I'd captured, because the CLI reports its own timing:

"timing": { "search_ms": 3998, "selected_ms": 0, "total_ms": 4041 }

3998 of 4041ms happened inside the daemon, on the other side of the socket. The subprocess accounted for 43ms of a 4-second call. The two implementations weren't a fast transport and a slow transport. They were two different questions asked of the same backend: mem context over-fetches to the 50-cap, resolves vault membership, then runs the cross-encoder rerank, and cmd:'prompt' does not.

Two other things fell out of measuring instead of theorizing. mem context without a vault fails closed and loud: error: the following required arguments were not provided: --vault <VAULT>. And an empty query dies loudly inside the engine with empty query — pass a non-empty search string rather than quietly returning the whole store.

Instrument under both paths before deleting the execFile call

When you find two implementations of one operation and one is slower, the transport is the decoy. Instrument the layer underneath both before you delete either: the subprocess, the HTTP hop, the extra serialization are all things you can see, which is exactly why they attract blame.

If the shared backend already emits timing, read it before you write a benchmark. Mine did.

Source: Node execFile was 2.7x slower: the spawn cost nothing by Chad Priest, from Building Vodou in Public.