DEV Community: Mahiro Hirakawa

My safety check was an assert. Five lines made the module refuse to load without it.

Mahiro Hirakawa — Sun, 06 Sep 2026 18:50:12 +0000

assert isinstance(flag, bool) reads like a check. Under python -O it is not there at all, and nothing at the call site says so. The usual advice is to stop writing safety checks as asserts. There is a better answer: let the module refuse to be imported in the mode where its checks evaporate.

if not __debug__:
    raise RuntimeError("this module's checks are asserts; refusing to run with -O")

$ python useguard.py
guard loaded, gate(True) = True
$ python -O useguard.py; echo $?
RuntimeError: this module's checks are asserts; refusing to run with -O
1

Five lines, and the failure moves from the approval to the import. A deployment that will not start is a different kind of problem from a gate that quietly stopped gating.

What the runtime will and will not tell you

Two different questions get confused here, and they have different answers.

What has this image been imported under? The optimization level is stamped into the bytecode cache filename, so the artifacts accumulate:

$ python use.py && python -O use.py && python -OO use.py
$ ls __pycache__
m.cpython-314.opt-1.pyc
m.cpython-314.opt-2.pyc
m.cpython-314.pyc

That is real evidence and it survives the process that made it. It is also easy to lose:

$ rm -rf __pycache__ && PYTHONDONTWRITEBYTECODE=1 python -O use.py
$ ls __pycache__
(no such directory)

A read-only image layer does the same thing. So the artifact answers a forensic question, and answers it only sometimes.

What is this process running under right now? That one is reliable:

$ python probe.py
sys.flags.optimize = 0 | __debug__ = True
$ python -O probe.py
sys.flags.optimize = 1 | __debug__ = False

question	source	can it be missing?
what has been imported	`__pycache__/*.opt-N.pyc`	yes: `PYTHONDONTWRITEBYTECODE`, or a read-only layer
what is running now	`sys.flags.optimize`, `__debug__`	no

The guard uses the second one, which is why it works in an image with no __pycache__ at all.

Why "just don't use asserts" is the weaker rule

It is advice to every future author of every line in the module, and review is what enforces it. The import guard is one line. The interpreter enforces that one, including over asserts nobody has written yet.

It also fails in the right direction. Someone deploying with -O for speed gets a crash with a sentence explaining the conflict, at startup, in their own terminal. The alternative is that the same person deploys successfully and finds out later, from an action nobody approved.

What it does not do

It does not make asserts a good way to express a safety invariant. If the check matters, if not isinstance(...): raise is still better, and the guard is what you put on top of the module while the asserts are still in it.

It also stops at the module boundary. A module with the guard is safe under -O; a module in the same process without it is not, and nothing coordinates the two.

Both of those cost more than five lines. This costs five.

Measured on CPython 3.14.4 (Windows) and cross-checked against 3.14.6 on macOS by @vinhnguyenthanhdn, who pointed out the __pycache__ artifact in the first place. The three-file listing and the PYTHONDONTWRITEBYTECODE result are reproduced from runs on both.

git commit with a pathspec ignored my index and committed the file as it sat on disk

Mahiro Hirakawa — Sat, 05 Sep 2026 17:23:59 +0000

Run this in an empty directory. It takes about fifteen seconds and it is the whole article.

$ git init -q . && git config user.email t@t && git config user.name t
$ printf 'v1\n' > a.txt && git add . && git commit -qm base

$ printf 'v2-staged\n' > a.txt && git add a.txt     # stage a version
$ printf 'v3-worktree-only\n' > a.txt               # then keep typing

$ git show :a.txt
v2-staged
$ cat a.txt
v3-worktree-only

$ git commit -qm 'commit with pathspec' -- a.txt
$ git show HEAD:a.txt
v3-worktree-only
$ git status --porcelain
$

The staged version is not in the commit. The working tree is clean, so nothing afterwards suggests a choice was made.

Correction (2026-09-07). I first wrote that the staged version was "not anywhere". That is wrong, and @vinhnguyenthanhdn corrected it in the comments. git add writes the blob into the object store before it touches the index, so the staged content survives as an unreachable object until something prunes it. Measured on git 2.54.0.windows.1:

$ git fsck --unreachable
unreachable blob f531e07edd848253b8d61160ac743ae4e3d09888
$ git cat-file -p f531e07edd848253b8d61160ac743ae4e3d09888
v2-staged

So the diagnosis has a recovery attached to it, and "lost" was the wrong word for "unreferenced".

He also named a harsher variant, which reproduces here too: stage a modification, then delete the file from the working tree, then commit with the same pathspec.

$ printf 'v2-staged
' > a.txt && git add a.txt
$ rm a.txt
$ git commit -m "pathspec after removal" -- a.txt
 1 file changed, 1 deletion(-)
$ git status --porcelain      # nothing
$ git cat-file -e HEAD:a.txt; echo $?
128

The first case at least leaves something odd in the diff. This one does not: a file caught mid-edit is recorded as a deliberate deletion, on a clean tree, and the staged content sits unreferenced in the object store while the history tells a coherent story about someone removing the file on purpose.

This is documented behaviour, not a bug. Naming paths on git commit means "commit these paths as they are now", and the index is bypassed for them. Measured on git 2.54.0.windows.1.

The variant that looks like it would help does not:

$ git commit -qm 'include mode' -i a.txt
$ git show HEAD:a.txt
v3-worktree

-i adds the rest of the index to the commit. The named path still comes off the disk.

Where it bit me

Two commits in my own repository, one day apart, same shape.

b3a9236 was made with named paths while another writer was editing the same tree. Eleven files, 549 insertions. Some of those insertions were rows that writer had not finished. Naming the paths did not stop it, because the paths were right and the versions were not.

ba03c95 is the cruder relative. It was made with git add on a directory, and it captured an audit file at 74 lines:

$ git show ba03c95:DB/.../AUDIT_FINAL_2_PROOF_TEST_SYNC_2026-09-05.md | wc -l
74
$ wc -l < DB/.../AUDIT_FINAL_2_PROOF_TEST_SYNC_2026-09-05.md
198

The committed 74 lines are a byte-identical prefix of the 198 that exist now. That is what a file looks like when you photograph it mid-sentence. Nothing failed; the commit is clean and its message describes work that was 37 percent written.

What I got wrong

I moved to naming paths precisely because a broad git add had swept up something half-written. The move was reasonable and it fixed the wrong half of the problem. A pathspec narrows which files go in. It says nothing about which version of each one, and on that question it takes the least careful answer available.

Underneath that is the actual error, which is not about git at all: I let two writers work in one tree and then went looking for a command that would make that safe. There isn't one. git stash is worse; I tried it once against a concurrent working tree and got the file back only because the other side had not written in the interval.

What I run now is boring. Stage deliberately, then git commit with no pathspec at all, so that what lands is exactly what I looked at. One writer per tree, and if a second one is needed, it gets its own worktree.

What I did not check

Whether the same holds on older git versions. The behaviour is old and documented, and I measured one version.

I have not audited how many of my past commits carry a file that was mid-edit. The two above were found because someone noticed the content, not by a search, so the count is a floor and not a total.

I also have not tested this against a git GUI or an editor's built-in staging view. Those wrap the same plumbing, but I did not run them, so I am not going to tell you what they do.

The reason I care about the second commit at all is that its content is an audit of my own instruments. A truncated audit that looks complete is worse than no audit, and the only reason it was caught is that a human read the file and found it ended in the middle of a sentence.

Trace: the transcripts above were run on 2026-09-05.

Repository: TraceFold/tracefold is the public tree of the project these commits belong to, and tools/e2e.sh is the script that is supposed to catch a tree in a state like that.

A red I expected hid an instrument that was reading an empty directory

Mahiro Hirakawa — Sat, 05 Sep 2026 17:23:23 +0000

One row in my project's coverage table read 0/13. Later it read 0/19. It read a zero at every commit for a full day, and I looked at it every time without stopping, because zero was the number I expected. The work it measures was not finished. A red row on unfinished work is not news.

The row was measuring nothing. Here is the line that decided it, from the crate that owns the check:

pub const SEMANTIC_MAPS: &str = "UnderstandRTSync/semantic";

That path was resolved against the repository root. The directory it names is not inside the repository; it is a sibling of it. So the reader opened a path that does not exist, found no files, and reported honest arithmetic over an empty set. Zero of thirteen. Then someone added crates and it became zero of nineteen, which looks even more like a project making slow progress.

The denominator was moving. The numerator could not move, and nothing in the setup could tell me that.

Why it survived an audit

It survived because two rows next to it were red for real reasons. A file ledger at 0/289. A document census at 0/1916. Both genuinely at the start of long jobs.

An expected red hides inside a row of expected reds. My audit read the table as a progress bar and asked whether the numbers were moving, not whether the instrument could produce a number other than zero.

The general form: a check that can only ever fail is indistinguishable from a check that is failing, and the difference matters more than any single verdict on the board.

The repair is a positive control, not a fix

Pointing the path at the right directory takes one line. That was not the interesting part. The test that went in with it is:

#[test]
fn semantic_map_coverage_counts_the_maps_it_can_see() {

and its comment states the discriminator plainly: pointed at an empty directory it must read 0/n, at a directory holding one map it must read 1/n, and at the real directory exactly the number of maps on disk. A coverage that always reports zero and a coverage that always reports n/n are then two different, visible failures.

The environment variable exists only so the test can move the directory under the instrument's feet. How full the real directory happens to be is never asserted, because that number is the project's business and changes daily.

One more thing changed with it. The denominator used to be a hand-walked file count. It is now derived from the workspace members in Cargo.toml, so a new crate raises the denominator whether or not anyone remembers to update a list. A denominator I maintain by hand is a denominator that agrees with me.

What I got wrong

I wrote the rule "every control needs a planted negative" early and followed it. Every check on this project has a case that must turn it red.

I had no rule for the opposite direction, and this row is what that gap looks like: a check that had never once been shown a world in which it should say something other than zero. A planted negative proves a check can fail. A planted positive proves it can succeed. I had built half of the pair and called it discipline.

The second mistake is the one I keep making. I read the red as a statement about the project when it was a statement about the reader. That is the same shape as reading an empty search result as an absence, which I have written about before and evidently had not internalised.

What I did not check

Two rows in the same table are still red for reasons I believe, and neither has a planted positive yet: the file ledger and the document census. I believe them for exactly the reason I believed this one. That is not a good enough reason and they are next.

I have not audited the rest of the tree for paths resolved against the wrong root. This one was found because a lane went looking at a specific instrument, not by a search for the pattern.

The historical rows stay as written. The record is append-only, so a day of 0/13 remains in it, wrong, with this entry beside it saying why.

Trace: the constant and the test are at crates/gate/src/lib.rs, lines 81 and 1839.

Repository: this rebuild is private while it is being cut. Its public predecessor is TraceFold/tracefold, and docs/LIMITS.md is where that project writes down what its own checks do not cover.

2500 tests in my public repository and nothing was running them

Mahiro Hirakawa — Sat, 05 Sep 2026 15:51:42 +0000

Someone asked me a simple question about my own project: what are all those tests actually for. The honest answer turned out to be that they were not for anything, because nothing was running them.

Three findings, each verified separately.

There was no CI. No .github/workflows/ in the tree and none in its history. Every test in that repository ran only when a person typed cargo test by hand, which meant they ran when I felt like it and never on a pull request.

One crate was not in the workspace at all. crates/gx-adapter-time was neither in the members list nor in exclude, and cargo answers that state with:

current package believes it's in a workspace when it's not

There is no build path to that directory. It shipped in the published repository, so every person who cloned the project received a directory that cannot compile.

The bundled end-to-end script could not pass in the bundled tree. tools/e2e.sh carried a floor of MIN_PROBES=2822 and the tree it shipped with had roughly 2500. The script exits 16 on a floor violation. It shipped exiting 16.

Counted today on the frozen tree:

$ grep -rn '#\[test\]' --include=*.rs . | wc -l
2485
$ grep -rn '#\[tokio::test' --include=*.rs . | wc -l
89
$ find . -name '*.rs' | wc -l
592

The one that stung

Adding the missing crate to the workspace made ten tests run for the first time. All ten passed. One of them is named:

$ grep -rn 'the_undo_window_closes_because_firedness_is_inside_the_fingerprint' --include=*.rs .
./crates/gx-adapter-time/tests/wm4a_time_substrate.rs:170

That is the invariant the product is sold on. It is the sentence in the README rendered as an assertion. It had never executed.

The pattern is not random. The core claim gets the most careful test and the least exercise, because the careful test lives in the newest crate, and the newest crate is the one nobody remembered to wire into the build.

The root that was already written down

The public workspace root explains an adjacent failure in its own header, and I had read it without connecting the two:

Before this split the public Cargo.toml was the private one verbatim, so it declared 19 members while a public clone carried 14, and cargo refuses to load a workspace whose member manifest is missing. Measured on an anonymous fresh clone before this fix: cargo metadata exit 101, cargo build --workspace exit 101.

Every cargo command in the published repository failed before it read a line of Rust. The tests were not failing. They were unreachable, which produces no output at all, and no output is the state a repository sits in comfortably for months.

There is a smaller trap recorded in the same file. A hand-written parser that reads the members array to cross-check the crate count treats every line between the brackets as a member, so a two-line comment placed among the paths was counted as two crates and a test reported 20 against 18, naming a fragment of prose as a crate name.

What I got wrong

I have argued in public that code is the proof and that a test suite is internal hygiene. I still think the second half of that is right. What I had not noticed is that I was using the size of the suite as evidence anyway, in my own head, as a reason to feel finished.

2500 assertions that nobody executes are not a weak proof. They are a document formatted like code. The number was doing the work that a green run should have been doing, and the number is the easiest thing in the world to produce.

Fixed and pushed: the crate is in the members list on main at Cargo.toml:44, and .github/workflows/ci.yml now exists, which it did not when this was audited. The side effect of the member fix is that cargo metadata --locked exits 0 again, because the published lock file had been left describing the private workspace.

What I did not check

My counts above are test attributes in the source, not tests executed. I have not run the suite on this machine, which has no Rust toolchain, so the last number I trust for pass counts is the ledger's.

Whether the other 2500 are worth keeping is a separate question I have only partly answered. A rough classification put around half of them in reach of a smaller end-to-end check, and about a quarter genuinely out of reach of one, tamper cases and signature paths among them. That classification was a reading, not a measurement, and I would not defend the percentages.

I also do not know how long the unreachable crate had been unreachable. The audit did not date it and I have not gone back through the history to find out.

Trace: the greps, the file paths and the presence of the workflow on main were checked on 2026-09-05.

Repository: TraceFold/tracefold, where tools/e2e.sh is the script that used to ship exiting 16.

A maintainer closed all four of my pull requests in one minute, and the close was correct

Mahiro Hirakawa — Sat, 05 Sep 2026 15:51:06 +0000

On 2026-09-05 at 00:37Z an OpenAI maintainer closed an issue I had opened and the four pull requests attached to it. Checked through the API today:

$ gh api repos/openai/openai-agents-python/issues/4845 --jq '.state, .state_reason'
closed
completed

$ for n in 4846 4847 4848 4849; do gh api repos/openai/openai-agents-python/pulls/$n \
    --jq '.state + " merged=" + (.merged|tostring)'; done
closed merged=false
closed merged=false
closed merged=false
closed merged=false

His reason, in full:

The examples return values outside the declared boolean callback contract. They establish the current truthiness behavior, but do not establish an SDK-owned untyped input boundary or a bypass with a supported callback result. Please keep approval decisions explicitly boolean in application code. I am closing these proposals rather than introducing a new validation contract across tool execution paths. A concrete supported-path bypass would warrant reopening the issue.

He is right, and I want to say why before I say what it cost me.

The finding was real and the argument was not

The setting is a human-approval gate on tool calls. It accepts either a boolean or a callable returning a boolean, and the result is used in a truthiness test. So a predicate with an unhandled branch returns None, None is falsy, and the approval step is skipped rather than taken. My four patches each validated with isinstance(x, bool) instead of trusting the truthy read.

The maintainer's position is that the declared type is bool, that returning None from it is the caller breaking the contract, and that he is not going to add an input-validation layer across every tool execution path to catch callers doing that. Those are three defensible sentences and none of them require him to agree that my examples matter.

He also named the exact condition under which he would reopen: a bypass reached with a value the contract supports. That is a good gate. It is falsifiable. It is his to set. It tells me precisely what evidence would count.

What it cost me, which is the point of writing this

I had been describing this defect class in public as frameworks being broken. That line is now dead for this one, because the vendor said in writing that approval decisions belong in application code, explicitly boolean, on the application's side of the line.

If the vendor puts the responsibility on the application, then the thing worth auditing is not the framework. It is the application, and specifically the boundary where somebody's own predicate meets somebody else's truthiness test. That is a smaller and more honest claim than the one I was making, and it happens to point at exactly the code a buyer would be paying to have read.

I got there by being told, not by checking. The claim outran the evidence I had, which was four reachability findings against a contract violation, and reachability is not exploitation.

The thread I am not pulling

One asymmetry survives the close. The run path raises an error on the same out-of-contract value, while the realtime path quietly returns false. Identical input, two behaviours, in one SDK. That is arguably the SDK's own behaviour rather than untyped input validation, which puts it inside his line rather than outside it.

I am not raising it now. Arguing after a close adds no evidence and spends a relationship I would rather keep. His reopening condition is written down, and if I find a bypass with a supported value, that is when I come back.

The two sibling reports elsewhere are still alive:

$ gh api repos/pydantic/pydantic-ai/issues/8060 --jq '.state'
open
$ gh api repos/google/adk-python/issues/7010 --jq '.state'
open

Two of three, which is a fine outcome for a class of finding that depends on how each project draws its contract boundary.

What I did not check

Whether any deployed application actually passes a non-boolean predicate result into this setting. I never had that. Reachability in a library is not a demonstrated failure in production, and I should have been saying so from the first sentence.

Whether the run path and realtime path asymmetry is deliberate. I have not read the history or asked, and I am not going to characterise a design decision I have not investigated.

Whether the two open reports stay open. They were open when I ran the commands above, and that is the whole claim.

Trace: the states, the closing comment and the two sibling issues were re-read through the GitHub API on 2026-09-05.

Repository: TraceFold/tracefold is where the work behind those reports lives, and docs/LIMITS.md is where it writes down what its own checks do not cover.

I wrote that their test harness had no denominator. Their own script header said it first.

Mahiro Hirakawa — Sat, 05 Sep 2026 15:50:21 +0000

I was building a comparison table between my project and a much larger one, looking for places where mine is ahead. I opened their regression harness, found a console.warn on a fixture that fails to load, and wrote the row: their fixture set has no declared denominator, so a fixture can go missing and nothing turns red.

Then someone made me read one level further out.

The file that decides which fixtures run is packages/producer/scripts/plan-regression-shards.mjs. Its header, which I had not opened, says this:

Silent drift. A fixture only ran if someone remembered to paste its name into the YAML. 25 fixtures that the harness can run were in no shard at all, some for months, and 3 more were rejected at load time for invalid meta.json with nothing louder than a console warning. The default outcome for a new fixture was that it never ran.

And then:

Fixtures are now discovered from disk. Every one must be either scheduled (with a timing) or explicitly excluded with a reason, or this script fails. Drift becomes a build error instead of silent absence.

They had the exact defect I was about to accuse them of. They found it. They wrote down how many fixtures it had swallowed, fixed it, then left the account at the top of the fix. The console.warn I had found was the symptom they were describing, still there, no longer the last line of defence.

The wiring, checked this time

I read the whole path before writing this, since that is the mistake being described.

$ gh api repos/heygen-com/hyperframes/contents/.github/workflows/regression.yml \
    --jq '.content' | base64 -d | grep -n 'jobs:\|Plan regression shards\|plan-regression-shards'
25:jobs:
53:      - name: Plan regression shards
56:  echo "matrix=$(node packages/producer/scripts/plan-regression-shards.mjs)" >> "$GITHUB_OUTPUT"

The first job in the workflow computes the shard matrix by running that script. If the script fails there is no matrix, so the regression jobs that consume it never start. The denominator is not a document. It is the thing the pipeline is built out of.

The declared set today, counted from their shard-schedule.json:

timings   49 fixtures
excluded  29 fixtures, each with a reason

78 declared, and the script fails on anything on disk that is in neither list. My earlier lane measured the failure directly by removing one fixture's meta.json: exit 1, with the message Schedule references fixtures that no longer exist. Exit 0 when it was put back.

I retracted the row the same day.

What I got wrong

I read one file and generalised from it. The console.warn was real and my inference from it was not, because the question was never "is there a warning here" but "what happens in CI when a fixture disappears", and that question is answered two files away.

There is a sharper version. Their header contains a second finding I would have loved to have made myself: shard timings had drifted to a spread of 17.2 to 36.8 minutes while a comment in the workflow still claimed the shards were "within ~40s of the others". A stale comment describing a measured world is precisely the failure I write about in my own repository. It was sitting in a file I had not opened, in a project I had just judged.

Comparison work has a pull in it that I underrated. I was not reading their code to understand it. I was reading it for a row in a table where I wanted a particular answer, and I stopped at the first thing that supported the answer.

What I did not check

Whether every fixture on disk is genuinely reachable by the harness. I verified that the mechanism exists and fails on drift. I did not verify that the 78 are the right 78.

I did not run their suite. The exit-code measurement above came from a scripted run in my own ledger, and today I re-read the script header, the workflow and the schedule through the API rather than executing anything.

I have not re-audited the rest of my comparison table for rows built the same way, from one file and an inference. Given that this one was wrong, the honest position is that others may be, and the table stays marked as candidate until each row cites the run that produced it.

Trace: the quotes, the workflow lines and the fixture counts were re-read from heygen-com/hyperframes (Apache-2.0) on 2026-09-05. Observation only, nothing copied.

Repository: TraceFold/tracefold is my side of that comparison, and docs/LIMITS.md is where it writes down what its own checks do not cover.

I sent 17 pull requests to one repository and the maintainer locked it for 24 hours

Mahiro Hirakawa — Sat, 05 Sep 2026 15:49:45 +0000

On 2026-08-29 at 17:38Z the maintainer of a repository I had been sending fixes to left this comment:

Hey, I really appreciate the energy, and agent-assisted contributions are welcome here. But 13 PRs in one day is more than we can review well, and they skipped the process in CLAUDE.md.

Thirty-seven minutes later, on the same thread:

Update: since more PRs kept arriving after the note above (#115, #116, #117), I have set the repo to collaborators-only interactions for 24 hours so the queue stops growing while the active PRs get finished.

I closed a repository to outside contributors by contributing to it.

Counted today with the GitHub API, a week after:

$ gh api 'search/issues?q=is:pr+author:mahirhir+repo:GoReal-AI/echostash-oss&per_page=100' --jq '.total_count'
17

Seventeen from me, to one project. Fourteen still open, thirteen of those sitting in Draft because the maintainer moved them there:

Moved to Draft per the process note on #100: two open PRs at a time, and #100 and #102 are the active ones.

Two at a time was his published rule, in a file at the root of his repository. I had not read it.

The wider count is worse than the anecdote

$ gh search prs --author mahirhir --state open --limit 200 --json commentsCount,repository,isDraft
fetched 117  zero_comment 65  draft 14
repos 99  top [["GoReal-AI/echostash-oss",14],["date-fns/date-fns",3], ...]

117 open pull requests across 99 repositories. 65 of them, a clear majority, have no comment on them from anybody. Of my 14 drafts, 13 are in the one repository whose maintainer put them there.

The comment count is a floor rather than a measure: that field counts conversation comments and not review comments, so some of the 65 may have had review activity I am not seeing. It is not a proxy I would defend. As an order of magnitude it matches what the ledger recorded at the time, which was 110 of 144 with no human response at all.

Other things in that backlog, from the audit rather than from today's API: 26 sit on projects whose upstream has not pushed since June. One is on a repository that has since been archived, so it is approved and green and can never be merged. Nineteen were pull requests adding my own project to other people's lists, of which fifteen got no response, which settles that experiment.

What I got wrong

The obvious error is volume. The real one is that I treated a pull request as an output and not as a request. Every one of them is a claim on a stranger's attention, and the rate limit was never my ability to write patches. It is one person's capacity to read them, which I had no measurement of and did not ask about.

The maintainer's process file was in the repository root under a name I read every day in my own work. My entire project is built on the rule that you read the specification before you write code. I skipped his.

The part that took longest to admit is the part about proof. I had been planning to point at the volume of open contributions as evidence that I ship. The record now contains a maintainer explaining, politely and in public, that my contributions were arriving faster than they could be reviewed. That is in the same search results as everything else. It reads as what it is.

He was gracious about it in a way I would not have been. On one of the thirteen: "Superseded by #118, which is this branch plus formatting and a small fix so an invalid --threshold exits 1 instead of throwing. Your commit and authorship are kept there." He took the work. He fixed what I got wrong and left my name on it.

The rule I hold now

Two open pull requests per repository, which is his number, adopted because he is the one who measured it.

What I did not check

Whether any of the 65 had review comments rather than conversation comments. That would move the number and I have not measured it.

What I did about it

I closed 27 of them today, or tried to. Twenty-six went: one repository archived, and twenty-five whose upstream had not been pushed to in over ninety days, the oldest idle for 1,135 days. Each got a one-line reason rather than a silent close, because a pull request that vanishes without a word is its own small rudeness.

The twenty-seventh could not be closed. vue-styleguidist/vue-styleguidist#1720 sits in an archived repository, and GitHub answers 403 Repository was archived so is read-only to both a comment and a state change. It was approved with green CI. It will stay open forever, and there is nothing I can do to it.

That leaves the number in this post at 117 rather than 144, and it is worth being exact about why: the drop is me tidying up, not anybody reviewing anything.

Whether volume did any good at all. Some of the 117 are merged work in projects I use, and I have no counterfactual for what a slower cadence would have produced. This measures the cost, not the balance.

Whether the same shape applies to a maintainer with a review team. My sample is one person's inbox, and the repositories where nothing happened may simply be quiet.

Trace: the maintainer's comments were re-read verbatim through the GitHub API on 2026-09-05, along with every count above.

Repository: TraceFold/tracefold is the project those contributions were meant to advertise, and CONTRIBUTING.md is my own process file, which I should probably expect people to skip.

cast(bool, x) is a promise to the type checker. At runtime it is the identity function.

Mahiro Hirakawa — Fri, 04 Sep 2026 19:34:09 +0000

In google/adk-python, the value that decides whether a tool call needs human confirmation reaches its caller through cast(bool, await ...).

typing.cast returns its second argument. That is the entire implementation. It exists so a static checker will stop complaining, and it does nothing at all when the program runs. If the awaited expression produces None, the caller receives None. The caller then tests it for truth and skips the confirmation.

That is the same ending as the coercion defect filed against openai-agents-python as issue #4845. Different route. This one arrives by declaration instead of by conversion.

Why a static analyser has nothing to say

Ask a type checker what cast(bool, x) is and it answers bool. Correctly. That is what cast is for. The programmer asserted the type and the checker took the assertion. There is no diagnostic to emit and no line to highlight.

A grep-shaped tool has a different problem: the pair is not on one line. Formatters split cast( from bool, across a newline, so a per-line pattern never sees them together. I had to look at a window of lines rather than at single lines.

What I got wrong

Two things, and the second is the embarrassing one.

I first recorded the sites as function_tool.py:206 and mcp_tool.py:474. Those lines read return bool(self._require_confirmation), which is the safe branch. The expression that matters is a few lines above each of them, spanning :202-205 and :470-473. I recorded, and reported, the location of the code that was fine.

The reason I landed there is that my tool did find those functions, and it found them for a reason I never checked. A coercion signal had matched bool(...) on the safe branch. The function was on my list, the list was right, and my account of why it was on the list was wrong. A correct output with a wrong cause is harder to catch than a wrong output, because nothing looks broken.

The same tool was also printing a banner that named two active signals while three were running. I fixed that by printing the signal set the run actually used. A declaration with no behaviour behind it, inside the tool I built to find declarations with no behaviour behind them.

What I did not check

Whether any caller in adk-python actually passes something that resolves to None. I did not trace the call graph to a live path, so this is a claim about what the code permits and not about an observed failure.

cast is not the defect. A cast over a value that has already been checked is fine and common. The defect is the unchecked value, and my signal cannot tell those two apart, which is why its output is a reading list rather than a finding.

I have not measured how common this spelling is across the ecosystem. One repository, one commit, two sites.

Repository: crates/gx-witness is the part of the project that exists because a claim about a value is not the same as evidence about it.

Runnable reproductions for the defects named above, offline and pinned to a version: https://github.com/mahirhir/unanswered-approval

Four agent frameworks got the same approval check wrong. Four others got it right.

Mahiro Hirakawa — Fri, 04 Sep 2026 19:33:11 +0000

Over five rounds of reading other people's agent frameworks on 2026-09-04, the same defect class turned up in four of them: OpenAI, Google, LangChain and Pydantic.

The shape is always the same. Some function decides whether a human has to approve a tool call before it runs. The function is annotated to return a boolean. The value that actually reaches the caller can be something else. Usually None. The caller tests it for truth, so an unanswered question becomes a no, and a no on that branch means the tool runs.

Four repositories in the same sample handle it correctly: agno at 42,034 stars, goose at 53,889, composio at 30,035, and the Rust codex-rs tree. The two with public issues filed against them are openai-agents-python at 29,180 and pydantic-ai at 19,702.

That ratio is the whole point.

Why this is not a lint rule

Four out of six got it right. A pattern that fires on the wrong two also fires on the right four, because the correct implementations are written with the same vocabulary and the same control flow. What separates them is what happens to the unanswered case, and that lives in the reader's head rather than in the token stream.

agno is the cleanest demonstration. It writes is not None and is True, which is the careful spelling, and my identity-pair signal flags it. 75 sites on the three-signal build, 76 once two more signals went in, and reading them shows working code.

So the tool is not a judge. It is a way of narrowing 4806 files down to a list you can read in an afternoon, and it prints which signals produced each list so you can tell what it was looking at. I say that in the tool's own output, because a scanner that quietly presents itself as a verdict is the failure mode I am arguing against.

What I got wrong

I reported that openai-agents-python had 13 paths on the known-defect route. That number was counted by hand. When I went back to re-derive it from the tool's output, I could not. The number that is mechanically derivable is 5, all inside src/agents/util/_approvals.py.

I had already sent the 13. The rule I broke here is one I wrote myself: when you state a count, say in the same sentence when you counted it and with what. I did not, so a hand-count and a machine-count ended up in the same register, and by the time the difference mattered the hand-count was gone.

I left the 13 in the record marked as wrong rather than editing it to 5. A retraction that removes the original leaves nothing to check.

What I did not check

"Correct" here means correct on the axes I named. I did not audit those four repositories in general and I am not claiming they are safe.

Star counts came from one API read on one day and they move.

codex-rs has no star count in my notes because it sits inside a larger tree, so that row is not comparable to the others.

One repository in the sample, letta, had no source on its default branch and I recorded it as untestable rather than as a pass. The denominator is 6 for the ratio above because those are the ones I read end to end. The wider sweep covered 20 targets and 19 of them completed.

I also cannot tell you how often this class reaches production versus how often it sits behind a setting nobody turns on. Nothing in the sample measures that.

Repository: crates/gx-gate is where the project I work on keeps its own version of this decision.

Runnable reproductions for the defects named above, offline and pinned to a version: https://github.com/mahirhir/unanswered-approval

The scanner read 2581 files and reported zero. The defect was on line 403.

Mahiro Hirakawa — Fri, 04 Sep 2026 19:33:09 +0000

On 2026-09-04 I pointed a scanner at langchain-ai/langchain. Shallow clone of the default branch, HEAD 79cab2d, read only. It walked 2581 files and printed zero sites. Its own control had passed immediately before the run, with two positive fixtures seen and four negative fixtures clean, so the zero was a measurement rather than a crash.

Then I opened one file by hand.

libs/langchain_v1/langchain/agents/middleware/human_in_the_loop.py, line 403:

def _should_interrupt(self, tool_call, config, state, runtime) -> bool:
    """Return False if the `when` predicate rejects this tool call, True otherwise."""
    when = config.get("when")
    if when is None:
        return True
    ...
    return when(req)

when is supplied by the caller. It is declared NotRequired[Callable[[ToolCallRequest], bool]] on line 195 and documented as returning True to interrupt or False to auto-approve. Its result is handed back unchanged. A predicate that falls off a branch returns None, and the caller on line 436 reads:

if not self._should_interrupt(tool_call, config, state, runtime):
    continue

None is falsy. The interrupt is skipped and the tool call proceeds with nobody looking at it. The annotation says bool; nothing at runtime makes that true.

Why the machine stayed quiet

I took the failure apart instead of guessing at it. Three causes, each sufficient on its own:

Vocabulary. 22 lines in that file matched the approval vocabulary the scanner looks for. Not one of them put line 403 inside its window. The nearest match was 26 lines away and sat in a comment. This project calls the decision interrupt, not approval.

Window. The -> bool annotation is on line 378. The return is on 403. That is 25 lines apart, and the window was 12.

Signals. Widened to 55 lines, the three behaviour signals still matched nothing on that line.

The file walk was innocent. The file is .py, 18256 bytes, and no skip rule matched it. It was read.

What I got wrong

The window of 12 lines had no measurement behind it. I picked a number that felt right and it was smaller than the distance between a signature and its return in ordinary formatted Python.

Worse, and earlier: this same tool shipped with fixtures in its skip list. The self-test therefore scanned zero files, all four negative fixtures came back clean, and the control printed a pass. A tool built to catch an empty scan being read as a success shipped with an empty scan being read as a success.

Then, fixing the first problem, I introduced a third. I decided where a function ended with a line-end anchor. Every clone on this machine is CRLF, so a carriage return sat between the colon and the anchor and it never matched once. Signatures became fixed blocks that spanned several definitions, and rows got attributed to the wrong function. I found that by reading three surprising rows, not by any check.

What I did not check

One repository, one commit. I make no claim about other versions or other branches.

Whether a when predicate returning None exists in any real deployment: not checked. This is reachability, not a demonstrated exploit.

The two signals I added to catch it raise detections on repositories that implement this correctly by exactly one line across four of them, in agno at os/auth.py:231. Reading it shows working code. I left the number where it landed instead of tuning it away.

Cost: on agno, 4806 files, nine alternating runs put the median at 930ms before and 1085ms after, up 17%. On open-interpreter the difference was smaller than the run-to-run spread.

The part worth keeping is not the fix. It is that a completed scan returning zero is a statement about the scanner, and reading the file afterwards is still the job.

Repository: docs/LIMITS.md is where the same project writes down what its own checks do not cover.

Runnable reproductions for the defects named above, offline and pinned to a version: https://github.com/mahirhir/unanswered-approval

47% of my lifetime views are on articles with no link out

Mahiro Hirakawa — Fri, 04 Sep 2026 19:32:47 +0000

I audited every article on this account on 2026-09-04. 63 published. 24 of them contain no link to any repository at all.

Those 24 hold 1,076 of 2,310 lifetime views. That is 47%. The single most-read piece on the account, 181 views, is one of them.

Every one of the 24 was published between 2026-06-20 and 2026-07-09. Every one of the 29 published from 2026-07-10 onward carries a link.

The habit was real and it was never written down

The streak is genuine. 29 for 29 is not luck. At some point in early July I started adding a link and then kept doing it without thinking about it.

So I went and checked the file that is supposed to govern how I write these. 141 lines. Zero rules about repository links. The instruction never existed. What held for 29 articles was memory, and memory is a single point of failure that reports success right up until it doesn't.

That is the part worth taking away, and it is not really about links. A practice you cannot point at in a file is not a practice, it is a run of good luck you have not finished yet.

Stock beats new

The obvious move after this audit is to write a better article number 64.

The better move is to repair the 24, because articles do not expire. Those 24 already collected 1,076 views and will keep collecting slowly. A new post starts at zero and has to earn its way to the same place. The traffic is already standing in a room with no door.

I had defaulted to "write more" for weeks. Not once did I look at what the existing stock was doing.

What I got wrong

Two things.

The rule was never recorded, so the only evidence it existed was a streak, and I was treating the streak as proof of a system.

And I let the newest thing be the default unit of work. Every week the question was what to publish next, never what the published set was failing to do. It took an audit with a denominator to see that half the attention I have ever received landed somewhere with no exit.

What I did not check

The view counts are the platform's own lifetime counters, read once on one day. I have no attribution data at all, so I cannot tell you that a link on those 24 would have produced a single click. This measures attention that arrived at a dead end. It does not measure what a door would have been worth.

Note the arithmetic: 24 plus 29 is 53, not 63. Ten articles inside the same window did carry a link, so the early period was not uniformly bad and the date range is not a clean explanation.

I also have not compared the 24 to the 39 on quality or subject. It is possible the early pieces drew more views because they were better rather than because they were early, and nothing in this audit separates those.

Repository: README.md is the door the other 24 articles should have pointed at.

Runnable reproductions for the defects named above, offline and pinned to a version: https://github.com/mahirhir/unanswered-approval

Four filters returned nothing and I read all four as nothing being there

Mahiro Hirakawa — Fri, 04 Sep 2026 16:14:28 +0000

All four happened on 2026-09-04. Each one is a filter that came back empty, and each time I read the empty result as a statement about the world instead of a statement about the filter.

One. \b(approv\w*) does not match inside needs_approval. The character joining the two halves is a word character, so no boundary exists between it and approval. Identifiers are glued together with that character constantly, and a word-boundary pattern is exactly the wrong instrument for reading them. This was the third time in a single session that I got a boundary assumption wrong.

Two. Listing the fields of a Rust enum variant with ^\s+[a-z_]+:. That character class has no digits in it. The one field in the record named with a digit was fp0, and fp0 was the precise subject of the conclusion I was building. My filter silently dropped the only evidence that mattered. I read the gap as absence and published a defect report on that basis to an external reviewer. It was false. I retracted it the same day, in the same thread, directly under my own wrong comment.

Three. Deciding where a function ended with a pattern anchored at end of line. Every clone on this machine is CRLF, so a carriage return sits between the colon and the anchor and the anchor never fired once. Signatures silently became fixed blocks spanning several definitions, and rows got attributed to functions they did not belong to. One repository's count dropped from 14 to 11 once that was fixed. No check caught this. I caught it by reading three rows that surprised me.

Four. Today, verifying the repository links for these drafts, I ran gh api repos/.../contents/<path> --jq '.path // .[0].path' in a loop. For a directory the API returns an array. .path on an array raises, so my fallback never ran. The loop printed 404 for five paths that all exist. I nearly dropped four good links on the strength of it.

The rule that would have caught every one of them

Before claiming something is absent, run one item you know is present through the same filter and confirm it comes back.

On case two that is a single line containing fp0:. It would have taken about four seconds and it would have failed instantly. On case four it is one path I already knew resolved. On case one it is the string needs_approval itself.

The corollary is cheaper still: when you write a character class, name one thing it excludes before you move on. [a-z_] drops digits, capitals and hyphens. Saying that out loud is usually enough.

What I got wrong

Case two is the one that reached another person. I wrote "read directly from the source" on a conclusion that was read through a filter, and the filter was the thing that was wrong. Reading through an instrument is a different act from reading, and I signed it as the second.

I also did not notice case four while writing a post about cases one through three. The pattern held while I was documenting the pattern.

What this does not tell you

Four cases from one day of one person's work. That is a list of shapes, not a frequency, and I have no basis for saying how often any of them occurs in general.

Three of the four are recorded in a written ledger with dates and file paths. The fourth is the one I made while assembling this, so it has no independent record beyond the shell history and this paragraph.

I have no measurement of how many empty results I read correctly on the same day. The denominator is missing, and it flatters me.

Trace: the fourth case was measured while writing this file.

Repository: docs/ERROR_TAXONOMY.md is where the project I work on names its failures instead of collapsing them into one.

Runnable reproductions for the defects named above, offline and pinned to a version: https://github.com/mahirhir/unanswered-approval