DEV Community: Patrick Hughes

I built refusals into my claim recorder

Patrick Hughes — Fri, 25 Sep 2026 16:05:15 +0000

I built refusals into my claim recorder

I added write-time refusals to my claim recorder on September 23, 2026. Six bad rows needed retraction.

The recorder now rejects malformed checks before accepting a claim. That makes the stored record easier to verify, but it cannot establish that the claimed work happened. A valid check still has to run and cover the result it names.

Canonical URL: https://bmdpat.com/blog/i-built-refusals-into-my-agent-s-data-recorder-2026

What did I change in the recorder?

I moved the malformed-check rejection to write time. My September 23 devlog records the change and six retracted rows. Those are separate actions: refusing a new bad row does not remove an older one.

The scheduled prompt changed too. run-agent.ps1 now appends the recorder command to every scheduled prompt. The devlog records 11 tests for that change. Those tests cover command flags, session naming, prompt insertion, and running the example through the recorder.

Which input does the recorder reject?

A file_contains check needs type, path, and pattern. The pattern is a regular expression. The September 23 regression involved a text key where the checker expected pattern. Changing the field name changes whether the checker can read the assertion.

I replayed that field mismatch through shape_refusal on September 24. This is the actual input and output, with an illustrative filename:

bad = {"type": "file_contains", "path": "report.md", "text": "PASS"}
print(shape_refusal(bad))

refusing a claim whose check can never verify: file_contains needs a non-empty `pattern` (a regex); the check has `text`, which the grammar does not read (rename it to `pattern`).
  check as this process received it: {'type': 'file_contains', 'path': 'report.md', 'text': 'PASS'}
  The grammar is config/claims/check.py (see config/claims/README.md).

The corrected shape returns None, meaning no shape refusal:

good = {"type": "file_contains", "path": "report.md", "pattern": "PASS"}
print(shape_refusal(good))

That call checks the definition. It does not read report.md. The full recorder also evaluates supported content checks at write time. The reproduction above tests only shape refusal, so I cannot use it to claim that a report contains PASS.

What does a valid claim still need?

A valid claim needs evidence that matches its wording. A check that a file exists can support an existence claim. It cannot, on its own, support a claim that the file contains a correct answer. The difference matters before you store the result.

The report can exist, open, and look finished while leaving the question unanswered. I described that distinction in checking the result beyond its schema.

I would narrow the claim to what the check covers. If I need the broader claim, I need a different check.

What happened to the six old rows?

The six bad rows were retracted, according to the September 23 devlog. The new refusal protects later writes. Retraction deals with claims already in the record. I need both, because changing the recorder does not establish the truth of its history.

That distinction also changes how I read a summary. If an earlier row claimed completion, a later correction has to travel with it. Otherwise the next agent can inherit the old answer after the underlying record has changed.

Keeping evidence in files gives me something I can reopen. It also leaves me responsible for reading the corrections. Writing agent memory to files gives the record a location. It does not make every sentence current.

What did the day's scan fail to establish?

The September 23 devlog recorded a green security summary beside a scan that read zero of five repos. The coverage count was enough to reject a clean-scan conclusion. It said nothing about what a completed scan would find.

This is why I keep the claim and its scope together. A malformed-check refusal addresses input shape. A coverage count addresses what was examined. A result check addresses what happened. Passing one leaves the others open.

The day's evidence stops there: write-time refusals, six retractions, and a scheduled-prompt change with 11 tests. I have no before-and-after count showing fewer bad claims across later runs. That outcome remains unmeasured.

Accompanying prompt

What the prompt does: It reviews whether one agent claim has enough evidence, without treating a well-formed row as proof of completion.

Copy/paste this prompt:

Role:
Review one agent claim against its evidence.

Context:
I will provide the claim, check definition, actual check result,
and intended scope. Treat missing inputs as unknown.

Task:
1. Identify missing or malformed check fields.
2. Compare the claim with what the check can establish.
3. Read the actual result and coverage, if supplied.
4. Name the evidence still needed for the full claim.

Output:
Verdict: supported, unsupported, or unknown.
Reason: cite the supplied evidence.
Scope: state exactly what the check covered.

Constraints:
Do not invent a result or run a command.
A valid row is not proof that a check ran.
Zero coverage cannot support a clean-scan claim.
Partial coverage cannot establish the whole scope.
Later corrections override earlier completion summaries.

Copy the block above.

See the agent operation I am building

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=i-built-refusals-into-my-agent-s-data-recorder-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Bonsai 27B vs Gemma 4 E4B: Which Should You Run?

Patrick Hughes — Fri, 25 Sep 2026 16:05:10 +0000

Bonsai 27B vs Gemma 4 E4B: Which Should You Run?

Ternary Bonsai 27B and Gemma 4 E4B target different memory tiers: choose Gemma 4 E4B for audio or a 4.5 GB Q4_0 weight file, and choose Ternary Bonsai 27B for a 7.2 GB deployed footprint when you need 27B-class math and language reasoning.

How do the architectures and memory footprints compare?

Ternary Bonsai 27B compresses a 27.3B base into a 7.2 GB deployed footprint using ternary weights, while Gemma 4 E4B runs 4.5B effective parameters with a 4.5 GB Q4_0 weight file. KV cache is extra. You choose Bonsai when that 7.2 GB footprint fits. You choose Gemma 4 E4B when the 4.5 GB weight file fits, or when the job needs audio.

Ternary Bonsai 27B derives from Qwen3.6-27B. The primary PrismML Hugging Face model card details ~27.3B ternary weights with a ~0.46B vision tower under Apache 2.0. GGUF Q2_0_g128 yields 1.71 bits per weight, an ideal size of 5.9 GB, and a 7.17 GB file (~7.2 GB deployed). FP16 takes ~54 GB, while 1-bit is 3.9 GB. PrismML documentation lists Q2_0 at 6.66 GiB. Conversely, the independent Google Gemma 4 model card notes Gemma 4 E4B carries 4.5B effective parameters (8B with embeddings). The Google memory table lists weights-only sizes: 17.9 GB for BF16, 8.9 GB for SFP8, and 4.5 GB for Q4_0.

Spec	Ternary Bonsai 27B	Gemma 4 E4B
Base params	27.3B ternary + 0.46B vision	4.5B effective (8B w/ embeddings)
Quantized size	7.17 GB (Q2_0_g128) / 6.66 GiB (Q2_0)	4.5 GB (Q4_0) / 8.9 GB (SFP8)
Weight size	~7.2 GB deployed	4.5 GB (Q4_0, weights only)
Context limit	262,144 tokens	128,000 tokens (512 sliding window)
Modalities	Text, image	Text, image, audio (up to 30s)

Which model handles your modalities and context length?

Gemma 4 E4B wins if your pipeline ingests audio clips, while Ternary Bonsai 27B wins if your prompts require a 262K context window. Gemma accepts text, images, and 30-second audio inputs across 128K context, whereas Bonsai processes text and images without audio support.

Gemma 4 E4B integrates a ~150M vision encoder and a ~300M audio encoder per the independent Google Gemma 4 model card, handling 30-second audio clips across 128K context. Ternary Bonsai 27B provides 262,144 context tokens for text and images per PrismML documentation, with mmproj adding +0.9 GiB in the docs (0.63 GB or 0.93 GB in the primary README). Bonsai gives double the context but omits audio. When inspecting agent context growth, use BMD, see what your agents actually did.

What do the benchmark numbers and local tests show?

Neither vendor tested these models side by side, so you cannot compare their benchmarks directly. Ternary Bonsai 27B scored an 80.49 average on an H100 in EvalScope, while instruction-tuned Gemma 4 E4B scored 69.4% on MMLU Pro and 42.5% on AIME 2026.

The primary PrismML Hugging Face model card reports EvalScope runs on an H100 (temperature 0.7, top-p 0.95, top-k 20): Bonsai scored an 80.49 average (94.6% of Qwen FP16 at 85.07) and 93.40 in math (versus 95.33). The 1-bit build scored 76.11. The table benched Gemma-4-31B, not E4B. Google's independent Google Gemma 4 model card lists instruction-tuned E4B scores: MMLU Pro 69.4%, AIME 2026 42.5%, LiveCodeBench v6 52.0%, GPQA Diamond 58.6%, MMMU Pro 52.6%, and MRCR v2 128k average 25.4%.

Our RTX 5090 log (5090-rig/benchmarks/2026-07-16-bonsai-27b.csv) measured Bonsai Q2_0_g128 at 3885.47 tok/s prompt processing and 123.37 tok/s generation. No E4B run is on file. Do not confuse E4B with the 26B model in our 26B comparison. Sizing local hardware needs care: see our 5090 local inference guide and track tokens with AI agent cost control.

How do you run each model locally?

You run Gemma 4 E4B directly in Ollama using the official model tag, whereas you run Ternary Bonsai 27B using llama.cpp with the custom ternary quantization file. Each loader uses different CLI flags and quantization formats suited to its respective architecture.

Pull Gemma 4 E4B using the tag on Google's Ollama integration page:

ollama run gemma4:e4b

The Google memory table identifies this build as {model-name}-qat-q4_0-gguf.

For Ternary Bonsai 27B, the primary PrismML Hugging Face model card gives the command:

./build/bin/llama-cli -m Ternary-Bonsai-27B-PQ2_0.gguf -p "Explain quantum computing in simple terms." -n 256 --temp 0.7 --top-p 0.95 --top-k 20 -ngl 99

The PrismML documentation names Ternary-Bonsai-27B-Q2_0.gguf, while upstream llama.cpp expects Ternary-Bonsai-27B-Q2_g64.gguf.

What should you do with this?

You should audit your hardware VRAM, verify your pipeline input requirements, and run a test prompt against the selected model tag. Selecting between these two models comes down to whether your workflow demands native audio processing or larger parameter capacity for complex text prompts.

If the job needs audio clips up to 30 seconds, or the 4.5 GB Q4_0 weight file is the size that fits, pull gemma4:e4b.
Deploy Ternary Bonsai 27B when the 7.2 GB deployed footprint fits and you need 262K context or the H100 math score. The model card says this build is past the ~6 GB per-app iOS budget.
Test generation speed with test prompts before wiring the model into active services.

Accompanying prompt

What the prompt does: This prompt evaluates your hardware specs and workload requirements to recommend either Ternary Bonsai 27B or Gemma 4 E4B.

Copy/paste this prompt:

Role:
Local AI Systems Architect

Context:
Evaluate local model deployments based on hardware limits, input modalities, and reasoning requirements.

Inputs:
- Available VRAM: __ GB
- Required Modalities: __ (Text, Image, Audio)
- Required Context Window: __ tokens
- Host Runtime: __ (Ollama, llama.cpp)

Task:
1. Check Available VRAM, Required Modalities, and Required Context Window.
2. If Required Modalities includes Audio, or only the 4.5 GB Q4_0 weight file fits, select Gemma 4 E4B.
3. If the 7.2 GB deployed footprint fits, Required Modalities has no Audio, and math or 262K context is required, select Ternary Bonsai 27B. Do not select it for a 6 GB per-app iOS budget.
4. Output the matching CLI command for the Host Runtime.

Output:
- Selected Model: Ternary Bonsai 27B or Gemma 4 E4B
- Rationale: Memory and modality fit
- Launch Command: CLI command

Constraints:
- Use only facts from published model cards.
- Do not assume audio on Ternary Bonsai 27B.
- Never exceed Available VRAM.

Copy the block above.

Weekly measured local runs: https://bmdpat.com/5090-reports

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=bonsai-27b-vs-gemma-4-e4b-which-should-you-run-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

I dropped the BMD kill date and named it flagship

Patrick Hughes — Wed, 23 Sep 2026 16:05:14 +0000

I dropped the BMD kill date and named it flagship

I officially dropped the /bmd kill date and named BMD the flagship software on 2026-09-21.

This decision required updates to the cannon, BMD Desktop, bmdpat, and pricing pages. I am moving away from arbitrary thresholds. The blog will now use a search-led model where dynamic GSC missed-click scoring replaces hard CTR thresholds. I have 0 confidence that the current security scan is actually working. The SecurityAnalyst returned a GREEN verdict with 0 P0 issues, but Gitleaks scanned 0 of 5 repos because the K: volume has been unmounted for 26 days. A check cannot verify what it cannot reach.

Why did my vault health check report so many stale tasks?

The vault health check flagged 92 of 104 root tasks as stale and 35 tasks as stalled for 14+ days because the check only reads folder roots. It does not recurse into subdirectories to find active work.

I found that several drafts were sitting in the Outbox past their 3-day TTL on 2026-09-21. I need to update the crawler logic. I also noticed the VaultAgent-GapResearch NotBefore date of 2026-09-22 closes a five-day Doctor blind window. This is a scheduling collision I have to fix.

What did I ship in bmd-desktop and bmdpat?

I shipped bmd-desktop version 3.47.15, which includes home polish, terminal fit, agent tabs, and build simplification. I also enabled the ability to send reports without a GitHub account via POST /api/bmd/feedback.

The following changes were merged on 2026-09-21:

Component	Version/PR	Key Changes
bmd-desktop	3.47.15 (PR 434)	Agent tabs, terminal fit, no GitHub account needed for reports
bmd-desktop	3.47.12 (PR 433)	Home panels, Triage all, Factory mode
bmdpat	PR 1807	Implemented feedback API to mail reports directly
bmdpat	PR 1805	Fixed Copy prompt button for in-app browsers

I also ran a quant ladder on the 5090. It measured Qwen3.8-27B (lmstudio-community) at Q4_K_M, Q6_K, and Q8_0 across 9 sweep rows in the desk corpus. If you are interested in how these weights affect performance, you can read my post on GGUF quantization explained or use my GGUF compare tool. This is useful for anyone running local LLM inference on consumer GPUs.

Who was I on 2026-09-21?

I killed a date instead of a product. The /bmd door carried a kill date of October 1 and targets I set myself, but on the morning of 2026-09-21 I saw that the signed installer only became public on 2026-09-20. A 14-day trial that starts now cannot finish before that date. So there was no result waiting at the end of the test, only a deadline. I dropped the date and said BMD is our flagship software. That is the opposite of the move I usually make, and it comes from the same place. When a measurement stops measuring, I stop trusting it.

Then I shipped all day. Three bmdpat merges, two desktop merges, a signed 3.47.15 a stranger can download, a Docker CI runner on the 5090 box, and a quant ladder that put three quants of one 27B model inside the same error bar.

And I did not plug in the K: disk. Day 26. The brief asked me for the 27th time, and my own security agent reported GREEN after reading zero of five repositories. I can take a gate off a product I believe in within an hour. I let a cable sit for four weeks. Both of those are me.

Accompanying prompt

What the prompt does: Updates blog writer templates to include structured input fields for consistent generation.

Copy/paste this prompt:

Role:
Blog Template Architect

Context:
I am updating my writing workflow to use a structured Inputs section in every post.

Inputs:
- Topic: __
- Key Numbers: __
- Broken Items: __
- Shipped Items: __

Task:
1. Create a draft structure for a devlog post.
2. Ensure the first sentence is a plain statement of what happened.
3. Include an H2 for technical details.
4. Use the Inputs provided to populate the body.

Output:
- A structured markdown post.

Constraints:
- No preamble or intro text.
- Use only the facts provided in the Inputs.

Copy the block above.

Weekly measured local runs: https://bmdpat.com/5090-reports

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Ten Merges, One Dead Disk, Four Fake Green Signals

Patrick Hughes — Wed, 23 Sep 2026 16:05:11 +0000

Ten Merges, One Dead Disk, Four Fake Green Signals

On 2026-09-19 I merged ten PRs into BMD desktop launch QA. First one landed at 01:53, last one at 21:41. Every one of them carried a test that failed before it passed.

Why did the morning brief read RED?

The morning brief on 2026-09-19 came back RED, but not for the reason I expected. jev-advisory never fired at 06:15. nightshift-supervisor ran, returned ok, and wrote nothing. And the storage volume has been dead for 24 days as of 2026-09-19.

That one dead disk blanks four separate systems: the leak scan, the autotrader report, the bmdpat world model, and the blog image publisher. All four still report green. Nothing is reading them, so nothing can fail. I would rather have four reds than four silences that look like reds cleared.

What actually shipped on 2026-09-19?

PR 397 merged as 1c1a3b8c, landing on main at 3.46.70. It fixed six first-run defects found during a signed 3.46.67 walk. PR 405 pushed main to 3.46.79, and that build got signed and went public in bmd-releases at 13:58 on 2026-09-19.

PR 406 squash-merged as 31af8d62, main at 3.46.80. It adds a rolling verification attempt budget, durable acceptance, and a project note guard. PR 407, card BMD-135, squash-merged as 4bc02169 once CI run 35481443734 went green.

Then PR 408 caught something worse. Acceptance found a focus defect in Deploy. A five-second refresh was replacing the list and taking keyboard focus with it, dropping a keyboard user to BODY. I pushed the fix as db47725f, bundle 3.47.1, with a REGRESSION block attached. CI run 35484546000 is queued as of 2026-09-19.

One card from problem-scout scored a 10 out of 10 on 2026-09-19. It says Exchange Web Services goes dark tenant by tenant starting 1 Oct 2026.

What did the overnight machine actually do?

The queue sweep for the night of 2026-09-18 into 2026-09-19 merged 0, held 0, escalated 0, across four launches ending at 03:35. Every card was ineligible because queue_control audit blocks both bmdpat and trading right now.

SecurityAnalyst reported GREEN with 0 P0 and 17 P2. But gitleaks and osv each scanned 0 of 5 repos and still called it clean. That is the same shape as the dead disk problem: a scan that finds nothing because it never looked.

Vault health is critical. 89 of 97 root queue tasks are 3 or more nights old. 15 Requests have sat 7 or more days unprocessed. brain-backup failed at 21:07 on 2026-09-19 because the GitHub mirror cannot fast-forward on diverged branches. The local copy still wrote that night. The GitHub mirror is the leg that has not moved since 2026-09-18.

The brain worker ran 51 seconds at 21:00 on 2026-09-19, wrote 187 characters, and produced no artifact. Same silent-success shape as the supervisor at 06:35. I keep hitting this: an agent that holds its output in a response instead of writing it to a file leaves nothing behind to check. That is most of the argument for give an agent a file, not a memory. A file is a thing you can open the next day. A memory is a thing that expired the moment the process exited.

What is the blog repair loop actually stuck on?

Three drafts are blocked on a worktree publish blocked error, because asset publish routes through brain think --heal before it can go live. grace_status.py exits 0, and its own docstring defines exit 0 as do not heal. So think and the scheduled healer disagree about one state, and the disagreement is silent. I put it in a Request instead of forcing a rushed post through it.

I check the app before it reaches a person. I do not yet check my own blog record the same way. Nothing about this got fixed on 2026-09-19. That is why I keep coming back to verify what an agent actually produced instead of trusting a status code. A run that says ok is not a thing that shipped. That is also the whole point of BMD, see what your agents actually did: the point is not the claim, it is the log underneath it.

Who was I on 2026-09-19?

I kept one decision for myself and gave another one away. The signed build and the publish still wait on my word, and I gave that word once, for 3.46.79. What I stopped limiting was the checking. One bundled app release a day, no cap on PRs, merges, or CI runs. The old rule put the limit on the checks. That was the wrong place for it.

Ten desktop PRs merged between 01:53 and 21:41, each behind a test that failed first. Late in the day a keyboard check caught the Deploy list replacing itself every five seconds and stealing focus with it. Someone has to live with this software after the tests stop running.

Then I read my own blog record. Nothing went live on 2026-09-19. The healer tried to publish three times, failed three times, timed out on the rescue, and wrote that the obligation was met. I do not know yet if fixing the exit-code disagreement fixes the silence underneath it.

Accompanying prompt

What the prompt does: Forces an agent to prove a task completed by writing a checkable artifact instead of returning a status message.

Copy/paste this prompt:

Role:
You are an execution agent that reports work by producing files, not by describing them.

Context:
A prior run returned "ok" with no artifact and no way to verify the claim.

Task:
1. Run the assigned job to completion.
2. Write the full output to a named file on disk, not to chat or memory.
3. Print the file path and a byte count as the last line of output.

Output:
- The file path.
- The byte count.
- Nothing else on that final line.

Constraints:
- If the file was not written, say so plainly. Do not report success.
- Do not summarize the work instead of producing it.

Copy the block above.

Weekly measured local runs: https://bmdpat.com/5090-reports

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=ten-merges-one-dead-disk-four-fake-green-signals-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

A second machine will not run a bigger model

Patrick Hughes — Tue, 22 Sep 2026 16:05:10 +0000

A second machine will not run a bigger model

NVIDIA published Personal AI Router. The project calls it PAIR. It routes inference requests across the machines on one network. It drives Ollama and LM Studio. It runs on Windows 11, Linux and macOS, on x64 and arm64, and it marks Windows on ARM experimental. The product page offers version 0.1.1 and calls it beta.

I went looking for the licence. The product page did not give me one. The repository did. The README and the LICENSE file both name Apache 2.0, and that LICENSE file opens with "Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved." So the code is open, and you have to open GitHub to find that out.

Then I read the docs. Two lines in them change what I would tell anyone who owns two machines.

Short answer: a second machine does not raise the size of model you can run. It raises how many requests you can run at once. And on a mixed rig, work lands on the slow card about as often as the fast one. Canonical URL: https://bmdpat.com/blog/nvidia-pair-routing-ignores-vram

It routes whole requests

The README says it plainly:

"PAIR routes each independent request to one node. It does not pool GPU memory, combine GPUs into a larger logical GPU, shard one model across machines, or split an in-flight inference request between nodes."

Read that list slowly. No pooled memory. No bigger logical GPU. No model split across boxes. No request split while it runs. One request goes to one machine and finishes there.

The overview doc adds the other half:

"Nodes do not share models, so a node can serve a request only for a model it already holds."

Each machine loads its own copy. The model has to fit on the card that answers. So your ceiling is the ceiling of your best single card, before PAIR and after it.

That is the assumption I expect most readers to bring here. Two boxes, more VRAM in total, a bigger model. This software does not do that. It sends whole requests to machines that can already serve them.

The scheduler counts jobs, not cards

The known-issues page lists what the scheduler ignores:

"It does not consider GPU model, available memory, current utilization, measured latency, whether a model is already loaded, or how expensive a request looks."

The same page says why:

"Routing does not consider VRAM at all, because the scheduler counts workloads."

So the ranking input is queued job count. Not the card. Not free memory. Not how busy the card is right now. Not how long the last request took.

On a rig where every machine is the same, counting jobs is a reasonable rule. On a rig where one card is strong and one is old, it is not. NVIDIA writes the outcome down:

"On a mixed one, expect work to land on a slower node about as often as a faster one."

That is the vendor saying it, in the vendor's own docs. It is the clearest sentence in the project and it is on the known-issues page, which is the page nobody opens.

What this means for the rig you have

Most people reading this have one good card and one older machine. That is the case the quote above describes.

Add the second machine and you do not get a bigger model. You get a second place to send a request, as long as that machine already holds the model. If it cannot hold the model, it is not a place you can send that work at all.

Then the scheduler adds its own problem. It does not know which of your two machines is fast. It counts queued jobs and picks. So one prompt comes back quickly and the next one crawls, and nothing in the software is trying to fix that.

The gain is real but narrow. If several requests hit you at the same time, and both machines hold the model, they run side by side instead of in a line. One person typing at one keyboard sends one request at a time. That person gains very little.

What to do before you install anything

Work out the largest model your best card holds on its own. That is your ceiling with or without a router. The sizing desk does that part.
Check whether the second machine holds the same model. The docs say a node serves only a model it already holds, and the scheduler does not read whether a model is loaded. Load the same model on every machine you want in the pool.
Count how many requests you actually send at once. If the answer is one, routing gives you almost nothing.
If your two cards are far apart in speed, plan for the slow answer. NVIDIA already told you to expect it about as often as the fast one.

What I did not measure

I have not installed PAIR on anything. This post reports NVIDIA's own documentation, not my testing. Every quotation above comes from the project's LICENSE, its README, its overview doc, and its known-issues page. There is no timing here, no tokens per second, and no VRAM reading, because I ran no test.

My own two machines are the mixed case in the quote. One holds a strong card. The other is years older and already runs measurement work all week. A scheduler that counts jobs would send real work to that older machine and pollute a run I care about. So I am not installing it there, and I have nothing measured to say about it yet.

I also want to be fair to the project. It is version 0.1.1 and it says beta on the page. A beta that documents its own worst behaviour on a page called known issues is doing better than most. The problem is not the software. The problem is the gap between what the docs say and what a reader with two machines assumes.

If you run two machines at home, what did you expect the second one to buy you?

q4_k_m vs q8_0: which GGUF quant should you use?

Patrick Hughes — Mon, 21 Sep 2026 16:05:08 +0000

q4_k_m vs q8_0: which GGUF quant should you use?

Q4_K_M is the best choice for speed and memory efficiency, while Q8_0 is better if you need maximum fidelity and have enough VRAM to spare. On Llama-3.1-8B, Q4_K_M uses 4.58 GiB of space, whereas Q8_0 requires 7.95 GiB.

How do Q4_K_M and Q8_0 differ in performance?

Q4_K_M offers much faster generation speeds and a smaller footprint, but Q8_0 maintains higher accuracy by staying closer to the original FP16 model's perplexity and KL divergence. You trade a small amount of logic precision for significantly more headroom on your GPU.

The difference comes down to bits per weight. According to the llama.cpp quantize documentation, Q4_K_M is approximately 4.89 bits per weight (https://github.com/ggml-org/llama.cpp/tree/master/tools/quantize). Q8_0 uses roughly 8.5 bits per weight (https://github.com/ggml-org/llama.cpp/tree/master/tools/quantize).

I look at perplexity to see how much the model "stumbles" compared to the original. In a llama.cpp perplexity test on Llama-3 8B, Q8_0 achieved a PPL of 6.234284, while Q4_K_M was higher at 6.407115 (https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/perplexity/README.md). Lower is better.

An independent study in arXiv:2601.14277v1 shows that Q8_0 is a high-fidelity baseline, but its compression is weaker than 4 or 5 bit options (https://arxiv.org/html/2601.14277v1). In task benchmarks, the average score for Q4_K_M was 69.15 and Q8_0 was 69.41 (https://arxiv.org/html/2601.14277v1).

Which quant should I choose for my hardware?

Choose Q4_K_M if your GPU VRAM is limited or you need high tokens per second. Choose Q8_0 if you have plenty of VRAM and the task requires the highest possible accuracy from the model.

The choice depends on your specific bottleneck. If you are running local LLM inference on consumer GPUs, VRAM is your most precious resource.

Use this guide to decide:

Feature	Q4_K_M	Q8_0
Llama-3.1-8B Size	~4.58 GiB	~7.95 GiB
Generation Speed	Faster (71.93 t/s)	Slower (50.93 t/s)
Accuracy (PPL)	6.407	6.234
Best Use Case	High speed / Low VRAM	High fidelity / High VRAM

Note: size and speed rows are the Llama-3.1-8B run in the llama.cpp quantize README (https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/quantize/README.md). The perplexity row is the Llama-3 8B run in the perplexity README (https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/perplexity/README.md), the rows measured without an imatrix. They are two different measurements.

If you are unsure about the math behind these sizes, read my GGUF quantization explained post or use the quant compare tool to see how different weights affect your model.

How do I create these quants myself?

You can use the llama.cpp quantize tool to generate these files from an existing F16 or F32 GGUF. You just need to specify the target type and the number of threads for the process.

Run this command in your terminal:

./llama-quantize input-model-f32.gguf q4_k_m 8

The q4_k_m part is the quantization type. The 8 at the end is the number of CPU threads you want to use. To make a Q8_0 version, simply replace q4_k_m with q8_0. Be careful when using the --allow-requantize flag, as it can reduce quality compared to quantizing directly from 16-bit (https://github.com/ggml-org/llama.cpp/tree/master/tools/quantize).

What should you do with this?

Check your available VRAM using nvidia-smi before downloading a new quant.
Download the Q4_K_M version first to ensure the model fits and runs at your target speed.
Only move to Q8_0 if you notice logic errors in the smaller quant and have the extra GiB of VRAM available for context.

Accompanying prompt

What the prompt does: Helps you decide between specific GGUF quantization levels based on your hardware constraints.

Copy/paste this prompt:

Role:
Hardware-aware LLM Engineer

Context:
I am running local LLM inference and need to choose between different GGUF quantization levels (e.g., Q4_K_M, Q6_K, Q8_0) for a specific model size.

Task:
1. Analyze my available VRAM and the model parameter count I am using.
2. Compare the estimated memory footprint of Q4_K_M vs Q8_0 for this model.
3. Recommend a quantization level that balances generation speed (tokens per second) and model perplexity.

Output:
- A recommendation for which quant to use.
- An estimate of how much VRAM will be left for context window/KV cache.
- A brief justification based on the trade-off between precision and speed.

Constraints:
- Prioritize preventing Out-of-Memory (OOM) errors.
- Assume I want to maximize tokens per second unless I state otherwise.

Copy the block above.

Weekly measured local runs: https://bmdpat.com/5090-reports

My sandbox passed two tests. The full run was unproven.

Patrick Hughes — Sat, 19 Sep 2026 16:05:11 +0000

My sandbox passed two tests. The full run was unproven.

I asked for the failures in my published devlog to be fixed on September 17, 2026. The queue sweep had stopped before it could run its commands. Changing the provider order had not fixed that.

By the end of the repair, Codex could run shell commands and two tests inside the restricted Windows environment. The blog was live too. I still did not have proof of a complete later nightly sweep.

What stopped the queue sweep?

The failed sweep used Antigravity inside a Windows sandbox. Its launcher hit a desktop permission error. It could not start the child commands needed to inspect files, run Python, use Git, or test changes.

That put the failure before the work I wanted done. A model response could not tell me whether the queue had moved. The launcher had to run a command first.

I had already written about moving four jobs to Gemini. That post described September 16. Its unresolved sandbox failure was still a real task when I read it the next day. I wanted the defect repaired, then checked.

What changed in the sandbox?

Codex changed the default sandbox provider order to Codex, then Claude. The ordinary jobs moved to Gemini kept their routing. This was a repair to the restricted queue runner, not a reversal of every provider change.

The Windows restriction stayed in place. The repair did not grant the launcher more permission to create a desktop. It used a provider path that could run the required commands within the existing boundary.

The first command probe confirmed the low-integrity token and returned the expected Python output. Git ran too. Then the test probe found another problem: the test temporary directory pointed outside the sandbox.

The runner now puts that directory inside the sandbox alongside the other temporary directories. A fresh model-driven test run passed two tests with exit code zero. That was the evidence I had wanted from the launcher repair.

Why was that still short of a nightly result?

The two tests proved a specific path: start the agent, run commands, and execute tests inside the restriction. They did not exercise every queue task or prove that a later scheduled sweep finished.

The September 17 queue review made that limit concrete. It reviewed 98 cards. Two were done, 37 were held, and 59 needed a decision. Reviewing a card did not turn it into shipped work.

I have written about checking an agent's output before. This repair gave me another small result to keep separate from the larger claim. I could say the launcher ran commands. I could not say the entire nightly queue now worked.

What else reached a finished state?

The blog repair produced a public result I could open. The September 16 article went live on September 17 at 12:41 CT. The later readback found its title, image, accompanying prompt, and closing link.

Its writer also kept a usable draft before trying to deploy the image. An image delay would no longer discard the article. That mattered because the draft and its hosted graphic could finish at different times.

The missing showwork evidence archive was recovered too. The repair used the actual merge CI log, package build, document response, and committed receipt. The old local test count remained unverified. Recovering the archive did not make that earlier number trustworthy.

My September 17 record contains both the morning failure and the later repair. The later evidence changes what I can say about the launcher. It leaves the full scheduled run unproven. I wanted fewer reasons to intervene, and I still needed that next run to show whether the repair held.

Accompanying prompt

What the prompt does: Separates a repaired command path from a verified end-to-end scheduled result.

Copy/paste this prompt:

Role:
Review the evidence for an agent runner repair.

Context:
I will provide the failure log, repair diff, test output,
and any subsequent scheduled-run report.

Task:
Identify the original failure. List the changed behavior.
Match each success claim to the command or result proving it.
Check whether the full scheduled job ran after the repair.

Output:
Return the verified repair, the unverified outcome,
and the next concrete check needed to close that gap.

Constraints:
Do not count available or reviewed tasks as completed work.
Do not infer a full-run success from a launcher test.
Apply later corrections without erasing the earlier failure.
Do not include secrets or private paths in the summary.

Copy the block above.

See the BMD agent workflow.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

My agent roadmaps did not prove the work was done

Patrick Hughes — Sat, 19 Sep 2026 16:05:08 +0000

My agent roadmaps did not prove the work was done

I wrote more plans than I could prove finished on September 18, 2026. My devlog put new roadmaps beside an aborted nightly sweep. Both happened. One did not cancel the other.

Summary: On September 18, 2026 I published new agent roadmaps beside an aborted nightly sweep. The four plans listed 24, 25, 28, and 28 cards, and those cards are planning artifacts, not finished work. The same record had a delivered email to seven subscribers, a live blog post at 11:37, and no measured saved minutes.

A roadmap records possible work and its dependencies. It does not authorize an agent to start every task. It also cannot prove delivery or time saved. My September 18 record had evidence for some deliveries, but no measured saved minutes.

What did the roadmaps actually establish?

The Brain roadmap had 24 unassigned backlog cards. The website roadmap had 25 planning cards. AgentGuard had 28 unassigned backlog cards. The showwork plan had 28 child cards.

Those counts describe planning artifacts. I cannot add them together and call the result shipped features. An unassigned card still needs someone to take it. A dependency still needs the work on the other side of it to finish.

The plans also carried weekly windows or targets. That gave the work an order. It did not prove that the dates would hold. My devlog recorded the plans as published, which is a valid result for a planning task. It is a narrower result than finishing the work inside them.

The choice behind the plans was simple. I wanted work across Brain, BMD, AgentGuard, showwork, and the website to improve the other work. Each product still had its own job. The plans described possible work. They did not give every agent permission to start it.

That last sentence matters when a machine can turn a list into actions. I need the plan to explain what could happen. I need the owning task to say what may happen now.

What evidence did I have for delivery?

The AgentGuard v1.3.2 release email had a different kind of record. I approved it after a delivered owner test. The September 18 devlog says all seven active subscribers received the announcement. A retry sent no extra messages.

That is evidence about an email. It does not show that seven people installed the release. It does not show that anyone read the message or wanted another product. I can report the delivery without adding a demand claim.

The blog had a separate result. After several failed runs, Codex and the publisher reached a live article at 11:37. The live page was the outcome. The failed runs still belonged in the record because the publishing path needed them before it worked.

I had already written about checks that prove what my agents did. September 18 made the distinction concrete again. A published plan, a delivered email, and a live article each answer a different question.

What remained unproven after those results?

The nightly sweep for September 17 had aborted when its sandboxed process exited without a report artifact. That missing report stayed missing in the September 18 record. A new roadmap could not supply it.

The same record listed one completed Brain Worker task, zero rerouted tasks, and three skipped tasks. I cannot turn the skipped tasks into completed work by describing the whole run as productive. Each result needs to keep its own status.

My sandbox repair post covers the related boundary: passing command checks did not prove a later full run. I still need evidence from that full run before changing the claim.

The time-return report had no explicit saved minutes. That does not mean the work saved zero minutes. It means I did not have a number I could defend. I left the uncertainty in the devlog instead of using the size of the roadmap as a substitute.

Which check in your agent report proves the work finished?

Accompanying prompt

What the prompt does: Separates planned work, checked outcomes, and unmeasured claims in a daily agent report.

Copy/paste this prompt:

Role:
Review my daily agent report against the supplied evidence.

Context:
I will supply the report and the artifacts it cites.

Task:
Separate planning artifacts from completed tasks.
For each claimed outcome, name the supporting artifact and check.
Keep failed, skipped, and unmeasured work visible.

Output:
A table with claim, evidence, supported conclusion, and missing proof.

Constraints:
Do not execute roadmap items.
Do not infer delivery from an attempted send.
Do not infer saved time from task counts.
Use later corrections when records disagree.
Mark unavailable evidence as unverified.

Copy the block above.

See how I run the agent fleet: https://bmdpat.com/bmd

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Gemini first for four jobs. One run finished two tasks.

Patrick Hughes — Fri, 18 Sep 2026 16:05:09 +0000

Gemini first for four jobs. One run finished two tasks.

On 2026-09-16, I changed the provider order for four recurring jobs in my agent fleet. Digest, Brain Worker, Obsidian maintenance, and supervisor review would try Gemini before Codex and Grok.

A longer provider probe passed after the first timed out, using my existing Antigravity login. The September 16 ledger recorded one Gemini Brain Worker run with two named tasks finished. The queue sweep still had a separate Windows permission failure.

What did the routing change prove?

The first probe timed out. A longer probe passed. That gave me a working provider path to test against scheduled work. It did not tell me whether every job could finish its own work with that provider.

I needed the next receipt: a real run with named results. Brain Worker's report supplied two. It removed a retired dashboard from the queue sweep's eligible queues. It also stopped claims from being made against temporary scratch files. Both changes were maintenance work that the recurring job was supposed to handle.

I could point to those changes when I asked what the switch accomplished. The provider name alone would have told me much less. A configuration can say Gemini first while the actual job fails before it changes anything.

That distinction is why I keep coming back to verifying what an agent actually produced. I want a result I can inspect after the process exits. A successful probe is useful evidence about the probe. A completed task needs its own evidence.

Why did I reject the bigger completion count?

An earlier summary in the day's record said Brain Worker completed fourteen agent-doable tasks. The later correction was explicit: fourteen tasks were available to agents. Fourteen tasks did not finish. The report named two finished tasks.

Those sentences describe different amounts of work. Using that number in the next summary would give me credit for work still sitting in the queue. The correction belongs in the public account too.

This is also a reason to give an agent a file it can read again. My record had room for the correction. The next reader still has to apply it. Preserving both sentences without resolving their disagreement would just preserve the error alongside the fix.

For this day, I can name the two results. I cannot turn the rest of the available work into completed work because the provider changed.

What remained broken after the switch?

The queue sweep had failed on a Windows desktop permission. Moving four provider routes did not prove that problem fixed. I still had a failed sweep to deal with after I had a recorded Brain Worker run.

The supervisor also withheld approval for showwork, my project for checking agents' completion claims. Its local evidence archive was missing, despite passing remote tests. I could not count that review as approved.

I want routine work to continue without my hand on every step. On this day, I had one recorded Brain Worker run after my routing decision. I did not have a repaired fleet or a completed product checkpoint. Both limits belong beside the work that finished.

Accompanying prompt

What the prompt does: Checks the reported work behind a provider change and separates it from probes, available tasks, and unresolved failures.

Copy/paste this prompt:

Role:
Review an agent run against its recorded results.

Context:
I changed the provider order for recurring jobs. Read the supplied configuration change, run ledger, and task reports as evidence.

Task:
1. Name the jobs whose routing changed.
2. Separate a successful provider probe from a completed scheduled run.
3. List completed tasks only when the report names their results.
4. Apply later corrections to earlier counts.
5. Identify failures that the routing change did not prove fixed.

Output:
- Routing changed
- Reported results
- Work still unresolved

Constraints:
- Do not count available tasks as completed tasks.
- Do not infer a repaired system from one successful run.
- If receipts disagree, explain the disagreement.

Copy the block above.

See what BMD includes: https://bmdpat.com/bmd

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

143 pull requests. Zero dollars.

Patrick Hughes — Wed, 16 Sep 2026 16:05:07 +0000

143 pull requests. Zero dollars.

The weekly review for 2026-08-31 through 2026-09-06 counted 143 merged product pull requests: 73 on the storefront, 60 on the desktop app, 6 on the verification package, 2 on the dashboard, and 2 on the trading repo. Guardrail tests sat GREEN. The same week, Stripe still read zero MRR, zero checkout starts, and zero paid strangers.

Short answer: a software factory can raise output and still fail the only number that buys back time. Throughput is the tell, not the win. Canonical URL: https://bmdpat.com/blog/143-prs-zero-dollars-2026

What did the week ship?

The build side was strong. 143 merged product PRs against 92 the week before. Six blog posts went out. The test suite sat at 454 of 454. None of that is fake.

None of that is revenue either. The first-dollar gate on 2026-09-06 read zero on all three success paths: a stranger charge, a checkout start, or an affiliate event. The demand-test kill on 2026-09-03 had already fired at 0 paid orders.

I have a standing rule for this fleet. More output is only good when it reduces stress, increases freedom, protects the capital engine, or creates useful work for real people. If an agent loop creates more work than it removes, it is failing even when it ships artifacts. 143 PRs and zero dollars is that sentence with a scoreboard attached.

What does the industry number say?

VentureBeat reported Faros AI figures on the software-factory thesis. Task throughput per developer was up 33.7 percent. PR merge rate was up 16.2 percent. The incidents-to-PR ratio was up 242.7 percent. Bugs per developer were up 54 percent. Google's DORA research associated more AI adoption with worse delivery stability.

Those are not my fleet's incident counts. I am not going to pretend they are. They are the shape of the same mistake: counting the cheap half of the loop.

PostHog wrote on 2026-08-11 that agents write code faster than any human can review. Their own shop had agents writing about 70 percent of PRs, and humans still skimmed at least 80 percent of those. That is lights-on, not lights-off. The dunk on lights-off factories is that the review never gets cheaper at the same rate as the generation.

I already have a post about empty pages that passed every schema check. This is the money version of that post. Volume is easy to count. Value is not.

What do I refuse to count as success?

Merged PRs. Agent headcount. GREEN test grids. Blog-post volume. Tokens per second. Saved-rig counts that are not strangers.

I will count paid orders, checkout starts, and time returned. I will count a kill rule that actually fires. I will count a model that earns promotion on held-out work, not a model that merely loads.

If your factory's weekly review leads with PR count, you are grading the intern on words typed.

Accompanying prompt

What the prompt does: It splits a weekly agent report into throughput metrics and outcome metrics, and fails the week if only throughput moved.

Copy/paste this prompt:

Role:
You are scoring one week of agent output.

Context:
[Paste the weekly counts: PRs, tests, posts, incidents, revenue, checkout starts.]

Task:
1. List every number that measures volume of work.
2. List every number that measures a result a stranger paid for, or a result that bought back time.
3. Say which list moved.
4. Fail the week if only the volume list moved.

Output:
- Two labeled lists.
- One line: volume week, or outcome week.
- One line: what you will stop counting.

Constraints:
- Keep it short.
- Use the numbers in the paste. Do not invent a conversion.
- Zero is a result. Do not skip it.

Copy the block above.

If you want the next receipts from this fleet, the list is here: https://bmdpat.com/5090-reports

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Ollama JSON: Empty Results Are Not Failed Requests

Patrick Hughes — Mon, 14 Sep 2026 16:05:12 +0000

Ollama JSON: Empty Results Are Not Failed Requests

I want my local extractor to say when it failed. Returning an empty list after a broken request hides the one fact I need.

An empty result is valid only after the request finishes and the response passes the application's checks. A timeout, invalid JSON, or missing field must remain a failure. JSON syntax alone cannot tell you whether the model found every item in the source.

Canonical URL: https://bmdpat.com/blog/ollama-json-empty-result-vs-failed-request-2026

Where does this affect my local stack?

My local model routing post describes separate prose and structured-data jobs. This post tests a smaller part of that system: the code that accepts an extraction result before another task uses it.

On September 13, 2026, I tested the Python function below with fixed response objects. These are test fixtures, not captured model outputs or a model accuracy benchmark. They let me check the caller's behavior without loading a model, changing a quant, or spending another inference request.

The target job is deliberately small. Given a note, extract a list of names. A note with no names can produce an empty list. A broken response cannot justify the same result.

What does Ollama return around the JSON?

For this example I use the native /api/generate response shape. Ollama documents response as the generated text and done as the completion flag. Its format field accepts JSON mode or a JSON schema. The endpoint streams by default; this example assumes the caller requests stream: false.

There are two objects to check. The outer object comes from the API. The inner JSON text comes from the model. Parsing the HTTP body does not also parse the text inside response.

I would pass an object schema with a required names array whose items are strings. Ollama's structured output guide shows how to request a schema and then validate the returned text. I still need application checks before the list reaches a downstream task.

How do I keep an empty result separate from failure?

Here is the complete acceptance function for the small example. It expects the decoded API object after the caller checks HTTP status. Transport errors must propagate before this function runs.

import json

def accept_names(reply):
    if not isinstance(reply, dict):
        raise ValueError("API response must be an object")
    if "error" in reply or reply.get("done") is not True:
        raise ValueError("Generation failed or is incomplete")
    raw = reply.get("response")
    if not isinstance(raw, str):
        raise ValueError("Missing generated text")

    result = json.loads(raw)
    if not isinstance(result, dict) or set(result) != {"names"}:
        raise ValueError("Expected only the names field")
    names = result["names"]
    if not isinstance(names, list):
        raise ValueError("Names must be a list")
    if any(not isinstance(name, str) or not name.strip()
           for name in names):
        raise ValueError("Each name must be non-empty text")
    return names

Python's json.loads raises JSONDecodeError for invalid JSON. I let that exception reach the caller. I do not catch it and return [].

An empty list passes this function when the response contains the exact expected object. A missing list fails. A string in place of a list fails. A blank name fails. These are different inputs with different outcomes, even when the dashboard would otherwise show the same empty table.

Which cases did I test?

The passing fixtures contain an empty names list and a names list with Ada. The failing fixtures cover an API error, an incomplete response, absent generated text, malformed JSON, a missing field, the wrong field type, a blank name, and an unexpected extra field.

I also tested a decoded API value that is a list instead of an object, and a generated JSON value that is a list. Both fail. That matters because valid JSON includes values other than the object my application expects.

The fixture checks establish parser behavior. They do not establish that any local model will extract names correctly. Keep those results in separate test reports so a passing unit test cannot become an invented model score.

What can this check still miss?

A completed response can contain valid JSON and still omit Ada from a note that names her. This function would accept that empty list. To test extraction quality, use source fixtures with known expected names and compare the returned list against them.

Completion also deserves its own check. Keep the recorded stop reason with the response, as I explain in my stop-reason post. The function above checks the completion flag; it does not prove the model finished the intended task.

My practical rule is to preserve the failure at the boundary. Save the error category and enough redacted context to reproduce it. Let the caller choose a bounded retry or a visible failure. A blank result should mean the extraction returned no names, with its accuracy still subject to the task test.

Accompanying prompt

What the prompt does: Adds separate tests for valid empty output, request failure, and extraction accuracy in a local Ollama client.

Copy/paste this prompt:

Role: You review a local Ollama extraction client.
Context: I will paste the request code, parser, and schema.
Task: Find every path that converts failure into empty output.
Output: Show a small fix and tests for valid empty results,
malformed responses, request failures, and known source answers.
Constraints: Keep the existing storage format. Do not call a
model. Use fixed fixtures. Separate parser tests from accuracy
tests. Do not claim that schema validation proves correctness.

Copy the block above.

I publish local AI tests and failure reports in The 5090 Reports. Join the email list for the next report.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

My Local Blog Writer Drops Private Lines First

Patrick Hughes — Mon, 14 Sep 2026 16:05:08 +0000

My Local Blog Writer Drops Private Lines First

I changed what my local blog writer can read. On September 13, 2026, I made yesterday's daily log its source. That also gave me a concrete privacy problem: the same log holds public build notes and private details.

A local model can still put private text into a public draft. My pipeline removes matching lines before drafting and checks the result again before publication. These checks cover named patterns, so a clean result still needs editorial review.

Canonical URL: https://bmdpat.com/blog/local-blog-writer-drop-private-lines-2026

Why filter the source if the model runs locally?

My normal writer uses Gemma through Ollama on owned hardware. That controls where the normal drafting request runs. It does not decide what belongs on my public blog.

Ollama's FAQ says the company does not see prompts or data when users run locally. It distinguishes local execution from cloud models. That is a statement about inference. My application still decides which files enter a prompt and where the finished text goes.

The daily writer also has an availability fallback. I need to know which model answered, as I explain in local model fallback provenance. I apply the source filter before drafting either way. A failed local load must not change which source lines the writer receives.

Why drop a whole line instead of replacing a name?

The function is scrub_devlog in config/blog/devlog_source.py. It removes the frontmatter, skips a private section, and drops any line that matches a configured private pattern. The remaining lines become the source text.

I chose deletion because a replacement marker can become part of the draft. A model can copy that marker into a sentence and leave the reader with missing information. Removing the line gives the writer less material, but avoids supplying that broken sentence in the first place.

Here is a synthetic example. One line says a parser now rejects a missing field. Another line gives the machine's private contact details. I want the parser result in the source. I want the contact line absent. I do not need the writer to explain that something was hidden.

The tradeoff is blunt. If one line contains both a useful result and a private detail, the filter removes both. I accept that loss for this source path. If the missing result matters, I can write a separate public note that stands on its own.

What do the checks actually test?

The implementation uses Python regular expressions. Python's re documentation describes the pattern matching operations behind those checks. The patterns cover configured categories such as machine names, profile paths, private addresses, contact details, and financial amounts.

On September 14, I ran the existing test_devlog_source.py suite. All eight tests passed. The mixed-source fixture checks that private sections and matching lines disappear. It also checks that a public pull-request result and an expression of doubt survive.

Other tests check clean text, category detection, source-date selection, and the command's exit codes. They also check that the voice profile exists. A missing voice profile raises an error instead of silently giving the writer no profile.

That test count describes this module. It does not measure recall across every possible private fact. It does not prove a model cannot infer a detail from the remaining context. I have no such result to report.

Why check the output again?

The writer's devlog validation scans the title, excerpt, and body for private terms. The publisher scans those fields again before its API request. The publisher sends a matching draft to its rejected folder and records the failure.

I want that last check because source filtering and output validation inspect different text. The first checks the material I supply. The second checks what the model wrote, including text introduced during revision.

This is the same reason I separate valid empty results from failures in my Ollama JSON parser example. The downstream step needs an explicit outcome. A draft that fails a privacy check must not quietly become a publishable draft.

What remains outside this filter?

A pattern list cannot understand every private fact. It can miss an unfamiliar name, an indirect reference, or an address format absent from the list. Several harmless lines can also reveal something when read together.

I still review the finished article. I keep source removal, output checks, and editorial review as separate steps. The useful result from September 13 is narrower: the daily writer now receives fewer private lines, and the publisher has a concrete stop condition.

For a local writer fed by your own notes, start with that boundary. Test one mixed note. Check exactly which lines survive. Then test a generated draft that reintroduces a forbidden detail. Neither test needs a new model or a faster GPU.

Accompanying prompt

What the prompt does: It asks a coding assistant to test source filtering and output rejection with synthetic notes.

Copy/paste this prompt:

Role: Review my local drafting pipeline.
Context: It turns private daily notes into public drafts.
Task: Inspect source filtering and the final publication check.
Output: Give tests for a clean note, a mixed note, and a draft
that reintroduces a private detail. State what each test proves.
Constraints: Use synthetic data only. Do not read private notes.
Do not call a model, publish content, or change the policy.
List untested cases. Do not claim complete privacy protection.

Copy the block above.

Get the 5090 Reports by email.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

DEV Community: Patrick Hughes

I built refusals into my claim recorder

I built refusals into my claim recorder

What did I change in the recorder?

Which input does the recorder reject?

What does a valid claim still need?

What happened to the six old rows?

What did the day's scan fail to establish?

Accompanying prompt

Bonsai 27B vs Gemma 4 E4B: Which Should You Run?

Bonsai 27B vs Gemma 4 E4B: Which Should You Run?

How do the architectures and memory footprints compare?

Which model handles your modalities and context length?

What do the benchmark numbers and local tests show?

How do you run each model locally?

What should you do with this?

Accompanying prompt

I dropped the BMD kill date and named it flagship

I dropped the BMD kill date and named it flagship

Why did my vault health check report so many stale tasks?

What did I ship in bmd-desktop and bmdpat?

Who was I on 2026-09-21?

Accompanying prompt

Ten Merges, One Dead Disk, Four Fake Green Signals

Ten Merges, One Dead Disk, Four Fake Green Signals

Why did the morning brief read RED?

What actually shipped on 2026-09-19?

What did the overnight machine actually do?

What is the blog repair loop actually stuck on?

Who was I on 2026-09-19?

Accompanying prompt

A second machine will not run a bigger model

A second machine will not run a bigger model

It routes whole requests

The scheduler counts jobs, not cards

What this means for the rig you have

What to do before you install anything

What I did not measure

Related reading

Accompanying prompt

q4_k_m vs q8_0: which GGUF quant should you use?

q4_k_m vs q8_0: which GGUF quant should you use?

How do Q4_K_M and Q8_0 differ in performance?

Which quant should I choose for my hardware?

How do I create these quants myself?

What should you do with this?

Accompanying prompt

Related reading

My sandbox passed two tests. The full run was unproven.

My sandbox passed two tests. The full run was unproven.

What stopped the queue sweep?

What changed in the sandbox?

Why was that still short of a nightly result?

What else reached a finished state?

Accompanying prompt

My agent roadmaps did not prove the work was done

My agent roadmaps did not prove the work was done

What did the roadmaps actually establish?

What evidence did I have for delivery?

What remained unproven after those results?

Accompanying prompt

Gemini first for four jobs. One run finished two tasks.

Gemini first for four jobs. One run finished two tasks.

What did the routing change prove?

Why did I reject the bigger completion count?

What remained broken after the switch?

Accompanying prompt

143 pull requests. Zero dollars.

143 pull requests. Zero dollars.

What did the week ship?

What does the industry number say?

What do I refuse to count as success?

Accompanying prompt

Ollama JSON: Empty Results Are Not Failed Requests

Ollama JSON: Empty Results Are Not Failed Requests

Where does this affect my local stack?

What does Ollama return around the JSON?

How do I keep an empty result separate from failure?

Which cases did I test?

What can this check still miss?