DEV Community: HideyukiMORI

I Ban the Second Way of Doing Anything: What That Costs

HideyukiMORI — Sat, 05 Sep 2026 15:23:40 +0000

For a long time I believed a good reviewer was enough to keep a codebase honest. Then I counted how many times I had written the same review comment.

"There's already a function for that." "We read the clock through the port, not directly." "That's the third way we format a date."

None of these were bugs. All of them were the same failure: a meaning that had grown a second implementation path. And every one of them had passed review, because reviewers hold rules in memory, and memory is exactly the thing that doesn't scale.

So I adopted one rule and made it the constitution of everything I build since: every meaning gets exactly one implementation path, and a machine keeps it that way.

Not "prefer". Not "should". One path, and the build fails if you add another.

what the rule actually says

It sounds like a style preference. It isn't. It's a claim about where truth lives.

If there is one way to read the current time, then "does this code depend on the wall clock?" is a question with a mechanical answer: does it import the one adapter that's allowed to? If there are two ways, the question needs a human, and the human needs to remember both ways, and next month there are three.

The rule cascades. One way to read time means one adapter that's allowed to. One adapter means a build check that rejects now() everywhere else. A build check means someone has to prove it fires. A proof means a document that quotes real output. Every step is boring. Every step is the thing that was missing the last time I found a second path in review.

I tested how far this goes by starting five small repositories in six days — Kotlin, C#, Go, Rust, Java — each with the same constitution, the same rule IDs, the same demand that every rule declare whether a machine enforces it or a person does. Five languages, one rule, 159 merged pull requests. That's the evidence behind everything below.

what it costs

Be honest about this part or the rule is just a slogan.

Adding a rule touches three places. The rule text, the enforcement matrix that says how it's held, and the actual check. Change one and the other two drift. So I wrote a check for that too — the docs and the matrix are validated against each other in the build. Now adding a rule is slower than not adding it. That's the point, but it's also the cost.

The checks themselves become a codebase. The Java clock has about 1,800 lines of dependency-free conformance checking to guard maybe 5,000 lines of application. For a desktop clock. I am not going to pretend that ratio is comfortable.

You have to prove the machine is watching. A check you have never seen fail is not a check. I learned this the hard way this week: my determinism gate rejected the first violation I planted, I wrote "active" in the log, and a second probe walked straight through. The config file behind the gate was being loaded by no build script. A different, bundled rule had caught my first probe by accident. Proving each gate fires — and reading which rule fired — is a real tax, and you pay it every time you add one.

Some of it never becomes mechanical. Across the five repos, the rules that stayed with humans were always the same kind: "this name is too generic", "this invariant is now written in two places", "this shortcut bypasses the boundary". Rules about meaning. Every language's toolchain could hold structure — dependency direction, module boundaries, forbidden calls — and none of them could hold meaning. I write those rules down as "planned" or "impossible" and leave them to review. The rule about one path applies to the enforcement too: don't claim the machine holds what it doesn't.

what I got for it

The second path stopped appearing. Not because people got better — because the build got red.

And something I didn't expect: the rule catches me. The five repositories were built by one person with an AI agent. The AI drifts toward the second path constantly, because the second path is usually the locally obvious one. The machine doesn't care who wrote it. In one afternoon the checks caught a half-wired gate, a test suite that had never run, and a proof document that quoted expected output instead of real output. All three were mine.

where this is wrong

This is the part I'd want to read if I were skeptical, so here it is.

Throwaway code. If the repository will not exist in a month, the three-place sync is pure overhead. I don't do this for prototypes, and I don't do it for scripts.

Teams that can't agree on which way is the one way. The rule assumes a decision. If two people each believe their path is canonical, mechanical enforcement just automates the argument. Decide first, then enforce. I have the easy version of this problem: it's one person and an agent.

The rule can't fully hold itself. When I put the five repositories side by side, the word "active" — as in "this rule is actively enforced by a machine" — meant something different in each one. The C# repo counted "tests and review" as enforcement. The Java repo would have called the same thing "planned". The Kotlin repo left 41 of 56 rules out of its matrix entirely, as review obligations. Same author, same week, five definitions. The principle demands one path for every meaning, and the meaning of "enforced" had five. I only found that out by counting. So: the rule is still held by a person at the top, and that person drifts too.

It's six days, not six years. Everything above is what it looks like when you start this way. I don't yet have evidence for what happens when a codebase built like this is three years old and the checks have to change. My older PHP tools have a lighter version of the same idea and it has held, but that's a different rule set and a different claim.

the shape of it

If you take one thing: the rule isn't "one way". Everybody says that. The rule is one way, held by something that doesn't forget — and then being honest, in writing, about every place where that something is still a human.

The five repos and their enforcement matrices are public: nene-clock (Java) is the most complete, and its QUALITY_GATES.md is the matrix I mean — 46 rows, 31 fully mechanical, 8 partially, 6 planned, 1 impossible. The half-wired gate story is its own post.

Where does your codebase have two ways of doing the same thing right now — and who is holding the rule that says there should be one?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

My Gate Failed for the Wrong Reason: Nothing Was Loading the Config

HideyukiMORI — Sat, 05 Sep 2026 15:03:52 +0000

I planted a forbidden call in my core module, ran the build, watched it fail, and wrote down "determinism gate: active."

The file that defined that gate wasn't being loaded by anything.

This is a small Java 21 desktop clock (nene-clock, MIT) that I built in two days with an AI agent as a test bed for one rule: every meaning has exactly one implementation path, and a machine — not a reviewer — keeps it that way. Part of that rule is "core never reads the wall clock." I had wired forbidden-apis to reject Instant.now(), System.nanoTime() and friends by method signature, and I had a written proof that it worked.

What you'll get from this post:

how a gate can fail for the right rule and the wrong reason, with the exact lines
two more dead checks the same afternoon exposed, including a test suite that had never run
the three habits I now apply before I write the word "active" next to anything

What broke

My repo keeps a file called gate-proofs.md. For every custom check, it records a planted violation, the task that ran, the output, and confirmation that removing the violation turns the build green again. The idea is simple: a check I have never seen fail is not a check.

The first entry, P1, was for the determinism rule. I put LocalDateTime.now() in :core:application, ran ./gradlew check, and it failed. I recorded the proof and moved on.

Before starting the next piece of work I tried a second probe, this time with two different APIs:

long probe = System.nanoTime() + java.time.Instant.now().toEpochMilli();

The build passed.

Why

The forbidden signatures live in config/forbiddenapis/determinism.txt. The convention plugin that every module applies was supposed to hand that file to forbidden-apis. Here is what it actually handed over, from the diff of the fix:

// build-logic/src/main/kotlin/neneclock.java-conventions.gradle.kts — before
forbiddenApis {
    bundledSignatures = setOf("jdk-unsafe", "jdk-deprecated", "jdk-non-portable",
                              "jdk-internal", "jdk-reflection", "jdk-system-out")
    signaturesFiles = files(rootProject.file("config/forbiddenapis/base.txt"))
}

Only base.txt. The determinism file existed, was committed, was referenced in the docs, and was read by nobody.

So why did P1 fail? Because forbidden-apis ships bundled signature sets, and jdk-unsafe forbids methods that silently use the default time zone. LocalDateTime.now() happens to be one of them. My probe had been caught by a rule about locale safety, not by my rule about determinism. System.nanoTime() and Instant.now() are not in that bundle, so they walked straight through.

It got slightly worse. The :adapters:system-time module — the one place allowed to read the clock — has an override that removes determinism.txt from its own signature list, so the exception is visible in the build file. That override was a no-op. You cannot remove a file from a list that never contained it.

The gate had failed for the right rule and the wrong reason, and the proof I wrote down was true and useless.

The other two things that were dead

Fixing the wiring meant rerunning everything, and rerunning everything found two more.

The conformance engine's own tests had never run. My project-specific checks live in build-logic, which is a Gradle included build. Tasks in an included build are not part of the root check unless you say so. I had written "every rule has positive and negative unit tests" — true — and neither ./gradlew check nor CI had executed them once. The first time I ran ./gradlew -p build-logic test, two failed. One of them meant that a correctly written waiver comment was being rejected, and the test for it was green because it only asserted the rule ID, not the reason the violation was reported. Failing for the wrong reason, again, one layer down.

The proof document's first version quoted output I expected, not output I got. When I retook P1 from the real console, the excerpt did not match what I had pasted. I had written the proof from memory of what the tool should say. That is not a proof; that is a wish with a code block around it.

The fix

PR #27 (+238 / −59), merged the same day as Issue #26 was opened:

Build the signature list once and hand it to both the main and test forbidden-apis tasks. Tests reading real time are a determinism failure too.
Make check depend on :build-logic:test, so the engine's own negative proofs run on every gate.
Add a new rule, CNF-011: if a file under config/ is not referenced by any build script, validateConformance fails. "Placed but not wired" is now something the machine says, not something I have to remember to look for.
Retake every proof from real output and keep the wrong first version in the history.

The probe now fails like this — quoted from the console, not from my head:

Forbidden method invocation: java.lang.System#nanoTime() [現在時刻は WallClockPort / TickSource からのみ得る。
ここで読むと決定性が壊れる（ARC-007）]
Forbidden method invocation: java.time.Instant#now() [現在時刻は WallClockPort / TickSource からのみ得る。
ここで読むと決定性が壊れる（ARC-007）]
Scanned 17 class file(s) for forbidden API invocations (in 0.02s), 2 error(s).

(The messages are in Japanese because the repo is; they say "the current time comes only from WallClockPort / TickSource — reading it here breaks determinism (ARC-007).")

And CNF-011's own negative proof, P12, planting an orphan file:

CNF-011 config/forbiddenapis/orphan.txt
      — この設定ファイルを読み込むビルドスクリプトが無い。置いても効かない

"No build script loads this configuration file. Placing it here has no effect."

The registry now has 14 proofs. Each one records what was planted, what task ran, what the tool printed, and that the build went green again after the violation was removed.

One thing I want to be precise about, because I got it wrong in a first draft elsewhere: this is not a story about Java being special. My Go repo enforces the same rule with depguard at the package level, and my Rust repo makes std::time a name-resolution error with #![no_std]. Java's forbidden-apis just names the ban at the finest grain — a method signature — which is what let me keep LocalDateTime.now(Clock) legal in the one adapter that needs it. The wiring mistake could have happened in any of them.

What I'd tell past me

A red build proves that something rejected your probe. It does not prove that your rule did. Read the message. Check the rule ID. If the tool that fired isn't the one you were testing, you have no proof yet.

A test that asserts only the rule ID will happily pass on the wrong rejection. Assert the reason, or at least assert on a fixture that can only fail one way.

Paste output; never type it. The moment I wrote what the tool "would" say, my proof document became documentation of my expectations. Those were exactly the expectations that were wrong.

And one for the toolchain rather than the person: presence of a config file is not evidence that anything reads it. If your gate has a config directory, write the check that fails when a file in it is orphaned. Mine is 53 lines plus a 46-line test, and it would have caught the original bug before I wrote a single proof.

Sources

Issue #26 — determinism.txt is not loaded by any build: https://github.com/hideyukiMORI/nene-clock/issues/26
PR #27 — the fix and the retaken proofs: https://github.com/hideyukiMORI/nene-clock/pull/27
gate-proofs.md §3 — the section titled "what we learned while proving", including the wrong first version: https://github.com/hideyukiMORI/nene-clock/blob/main/docs/quality/gate-proofs.md
The enforcement matrix, where this rule went from "active" back to honest: https://github.com/hideyukiMORI/nene-clock/blob/main/docs/QUALITY_GATES.md

Related posts from the same repo family: CI Paid for Itself on the First Run: 4 Repos, Real Bugs and My Generator Hid a Lint Error: CI Never Checks What Nobody Commits — both are the same shape: a green result that was measuring the wrong thing.

Which check in your build have you seen fail for the reason you think it fails for?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

Two AI Agents Shared a Todo File and a Line Vanished

HideyukiMORI — Sat, 08 Aug 2026 08:06:55 +0000

A line disappeared from my todo file.

Not corrupted. Not moved to the archive where completed items go. Just absent — present in one version, missing in the next, with nothing in between reporting a failure.

I run several AI coding agents in parallel, one per repository, and they share a single plain-text todo file as their cross-repo coordination point. I've written before about why plain text beat a SaaS board for this. This post is about the failure mode I hadn't accounted for.

The shape of the bug

Every writer to that file — a CLI, a terminal UI, and the agents themselves — does the same thing: read the whole file, change one line in memory, write the whole file back.

Three processes doing full-file read-modify-write on one path, with no locking anywhere.

That's a lost update, and it's textbook. If session A reads the file, session B reads it, A writes, then B writes, B's copy never contained A's change and now the file doesn't either. No error surfaces because nothing went wrong at the filesystem level. Both writes succeeded. One of them just described a world that no longer existed by the time it landed.

I had built a coordination mechanism whose entire job was to be shared, and left out the only part that makes sharing safe.

The part that was harder than the fix

Here's what I actually want to pass on, because the concurrency bug is the boring half.

When I noticed the file looked wrong, I did the obvious thing: diff it against the last backup. Which produced a wall of differences, because between those two points several agents had legitimately edited many lines — rewording items, compressing finished work, updating deadlines. Almost every line had changed somehow.

So "what's different" was useless. Nearly everything was different. The question I actually needed to answer was narrower: is there anything that exists in the old version and has no counterpart in the new one?

That's a set-difference question, not a line-diff question. And answering it meant picking a distinctive word from each candidate line — a repo name, an error string, a ticket number — and searching for that word across the current file. Most of the "missing" lines turned up immediately in reworded form. One didn't. That one was really gone.

A line diff tells you what changed. It doesn't tell you what's missing.

If your recovery instinct is diff old new, it will work beautifully on a file nobody edits and drown you on a file that several writers touch all day.

What I changed, and what I haven't

The honest status: the analysis is done, the fix isn't shipped. The candidates are file locking around every writer, or a modification-time check before each write that aborts if the file moved underneath you, or simply backing up more than once a day — because right now that daily snapshot is the only restore point I have, which is itself a finding.

I'm not going to pretend I've closed this. What I did do immediately was change my recovery procedure, since that cost nothing and would have saved the hour I spent chasing phantom deletions.

The same bug, one layer down

What makes this worth writing up rather than filing away is that I hit the identical structure the same week, in a product, in a database.

One of my apps had just gained a scheduler that sends payment reminders on a timer. Multiple workers, one queue of things to send — so exactly one worker must own a given run. The obvious implementation is: check whether the lock row is free, and if it is, claim it. Two lines of code, and a window between them where another worker can do the same check and reach the same conclusion. Same lost update, dressed as an invoice going out twice.

SQLite was one of the supported backends and it has no advisory locks, so the design couldn't lean on a database primitive. That constraint turned out to be a gift, because it forced the version that's actually testable: a plain table where acquiring is a single conditional UPDATE, and a first-time claim is a single INSERT on the primary key. No check-then-act window, because there's no gap between checking and acting — the database's own atomicity does the arbitration. Releasing verifies a holder token, so a worker can't release someone else's lock. A TTL reclaims locks from workers that died holding them. Six tests, verified against three database engines.

The todo file and the scheduler are the same bug at different altitudes. The database one got a rigorous fix because it was going to touch customers. The file one is still open because it only bites me.

I notice that's a real ordering, and also that it's how the file ended up unprotected in the first place.

Takeaways

If more than one process writes a whole file, you have a lost-update bug, whether or not you've seen it yet. Agent sessions count as processes. They're just processes that write convincing commit messages.

Make claiming a resource one statement, not two. Any "check, then act" pair has a gap in it, and the gap is where two workers agree they both won.

When you're recovering a multi-writer file, don't ask what changed — ask what has no counterpart. Pick a distinctive token from each old line and search for it in the new one. Reworded is not deleted, and a diff can't tell them apart.

The lock table, the single-statement claim, and its six tests are here: nene-clear#406. The todo file still has no lock.

What's the shared file in your setup that nobody has put a lock on?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

My Contribution Graph Broke and My Data Was Fine: Three Queries Told Me

HideyukiMORI — Wed, 05 Aug 2026 16:48:23 +0000

My GitHub profile said Something went wrong where the contribution graph should have been. The activity timeline underneath it was empty too.

Two explanations arrive uninvited when that happens. Either something is wrong with my account, or GitHub is having an outage. Both are worth ruling out before you start refreshing and hoping, because they imply completely different responses — and as it turned out, neither was true.

What follows is the whole diagnosis. It took about five minutes and three queries, and the method generalises well past this particular breakage.

Find the actual request behind the broken thing

A rendered page that fails tells you almost nothing. You want the specific call it makes.

The contribution graph is drawn from contributionsCollection in GitHub's GraphQL API, so I asked for it directly:

gh api graphql -f query='{
  user(login: "your-account") {
    contributionsCollection { contributionCalendar { totalContributions } }
  }
}'

That returned 502 Bad Gateway after about 11 seconds. Three attempts, three 502s.

This is already worth something. A reproducible 502 with a long latency is a server-side timeout, not a browser problem — which means refreshing, clearing cache, and trying another browser are all guaranteed to waste your time. I stopped doing any of that immediately.

But "one query is slow" doesn't yet tell me whether my account is broken.

Control 1: is the account itself healthy?

Ask the same account for something trivial:

gh api graphql -f query='{ user(login: "your-account") { login createdAt } }'

Instant response. The account resolves, the record exists, GraphQL is happy to talk about it.

So the account isn't damaged or suspended. Whatever is failing is specific to the expensive query, not to me as a user. One suspect eliminated.

Control 2: is GitHub itself down?

Run the same expensive query against a different account. I used Linus Torvalds, because it's a public account with a lot of activity and the query is read-only:

gh api graphql -f query='{
  user(login: "torvalds") {
    contributionsCollection { contributionCalendar { totalContributions } }
  }
}'

Returned fine.

This is the control that matters most, and it's the one people skip. Same endpoint, same query shape, different input — and it works. So the aggregation machinery is up. The failure needs my data to happen.

Second suspect eliminated. By now I know the problem lives in the intersection of this query and this account, which is a much smaller place to look than "GitHub is broken."

Control 3: shrink the input until it works

The default contributionsCollection window is a trailing year. So I asked for a week:

gh api graphql -f query='{
  user(login: "your-account") {
    contributionsCollection(from: "2026-07-12T00:00:00Z", to: "2026-07-19T00:00:00Z") {
      totalCommitContributions
      totalPullRequestContributions
      restrictedContributionsCount
    }
  }
}'

It returned immediately: 2,024 contributions in seven days — 560 commits, 541 pull requests, 457 in private repositories.

That single response answers both remaining questions at once.

The failure is a function of window size, so it's an aggregation timeout — the backend can't finish summing a year of this account's activity inside its own limit. And the underlying data is intact, because a seven-day slice of it just came back with real numbers in it. Nothing is corrupt or missing. Only the summing-up is failing.

A view that won't render and data that's gone are different problems, and one query tells them apart.

That distinction was the whole point of the exercise. If the data had been damaged I'd have had an urgent problem. Instead I had a cosmetic one, and I could go back to work.

Why it happened, and what I did about it

The cause is unglamorous: the day before, I'd pushed an unusually heavy burst of work across a lot of repositories at once. A trailing-year aggregation over a high-activity account spanning dozens of repos — private ones included — is simply an expensive thing to compute, and that burst pushed it past whatever budget the backend allows.

I want to be careful here, because I can't see GitHub's internals. That's the explanation most consistent with the evidence I have — window size determines failure, other accounts are unaffected — not something I confirmed from the inside.

What I did about it: nothing.

That was the actual decision, and it was only available to me because of control 3. Once you know the data is fine and the failure is a load-shaped timeout on a view, the correct response is to leave it alone and let the aggregation get cheaper as the burst falls out of the trailing window. There was no fix to apply. There was a wrong action available — filing a support ticket about data loss, or worse, "repairing" something — and skipping it was the win.

Eleven days later I ran the same year-long query again. It came back in 2.3 seconds with 21,253 contributions. It healed exactly the way the evidence said it would.

The method, minus my specific problem

Three questions, in this order, each answerable with one call:

Does a cheap request to the same subject work? If yes, the subject is fine and you're looking at a query problem, not an account problem.

Does the same expensive request work for a different subject? If yes, the service is fine and the failure needs your particular data.

Does the expensive request work on a smaller input? If yes, it's a scale or timeout issue — and crucially, you have just proved the data underneath is readable.

None of that is clever. What makes it useful is that each answer eliminates an entire class of cause, so five minutes of it beats an afternoon of refreshing the page and reading status pages that say all systems operational.

And the third question does double duty: it's the one that tells you whether you're facing an emergency or an inconvenience. That's usually the thing you actually need to know first.

What's the last "everything is broken" you had that turned out to be one query being asked for too much at once?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

I Made My First Sync Do Nothing on Purpose: Proving Idempotence Before It Can Hurt

HideyukiMORI — Wed, 05 Aug 2026 10:32:58 +0000

The most dangerous moment in a sync's life is the first time you run it.

I write my posts in a git repository and push them to a publishing platform through its API. The sync is one-way and it sends whole articles: fetch everything that's live, compare against what's in the repo, and PUT the ones that differ.

Think about what that does on run number one, when the repo has never been reconciled against live. Every article where my local copy is stale gets overwritten with the stale copy. All at once. On published posts that people have already read.

The usual mitigations are eyeballing a diff, or enabling it for one article first. Neither of them proves anything about the other articles. I wanted a first run whose correctness was structural rather than checked by hand.

Seed from live, not from your repo

The trick is almost too simple to write down: build the initial state of the mirror by downloading what's already published, not by using what you have locally.

If the mirror starts as an exact copy of live, then logically the first sync must find zero differences and change nothing. The first run becomes a no-op by construction.

And that turns the first run into an experiment. If it really does nothing, then my fetching, my normalisation, and my comparison all agree with the platform's own idea of what those articles contain. If it tries to write something, my comparison is wrong — and I've learned that before the write happens, on a run I already expected to be silent.

I called it canary zero: the canary that proves the machinery works by not moving.

Here's the real log from that first run:

⏭ 03-dev-building-a-typed-cms-for-business-data (id=4052442) 差分なし → no-op
⏭ 04-dev-building-self-hosted-business-tools-for-japan (id=4024309) 差分なし → no-op
⏭ 09-dev-disposable-tenant-demo (id=4099005) 差分なし → no-op
⏭ 11-dev-introducing-nene2-ai-readable-business-apis (id=3957239) 差分なし → no-op
⏭ 12-dev-mcp-should-not-touch-your-database (id=3989175) 差分なし → no-op
⏭ 14-dev-prod-fix-not-hostage (id=4188474) 差分なし → no-op
⏭ 16-dev-shared-todo-txt-human-ai (id=4168967) 差分なし → no-op

合計 PUT=0 no-op=7 新規skip=0 ファイル欠=0 不一致=0

Seven articles, zero writes.

I verified that something worked correctly by looking at a log where nothing happened.

Before even that, the sync had a dry-run mode — do everything except the write, and print what it would have sent. Idempotence is a property you want to demonstrate before the first PUT exists in your history, not after.

Normalisation is not preprocessing. It's the definition of "different."

The comparison only means something if you've decided what counts as a difference, and this is where the design earns its keep.

Mine is deliberately thin — line endings, trailing whitespace, runs of blank lines, and the quotes the platform strips from the frontmatter title:

def normalize(s):
    s = s.replace("\r\n", "\n")
    s = "\n".join(l.rstrip() for l in s.split("\n"))
    s = re.sub(r"\n{3,}", "\n\n", s)
    s = re.sub(r'(?m)^(title:\s*)"(.*)"\s*$', r"\1\2", s)
    ...

Then the dry run found something I would never have predicted.

The platform adds language labels to unlabelled code fences. I write a bare `, it looks at the contents, decides that's shell or plaintext or conf, and stores it that way. So the version I sent and the version it returns are permanently different, through no edit by any human.

The consequence isn't cosmetic. Every sync would see a difference, PUT the article, get back the labelled version, and see a difference again on the next run. A permanent write loop against published articles, driven entirely by an artifact of the comparison.

The fix is one line — ignore the fence label when comparing:

`python s = re.sub(r"(?m)^(`)[\w.#+-]*", r"\1", s)
`

Note what it does not do. It only affects comparison. What gets sent is still exactly what's in the repo, so the platform keeps applying its own labels and I never fight it. Two safeguards for one rule.

After that, the run went back to PUT=0 no-op=7. Zero churn, verified the same way as before: by nothing happening.

This is the part I'd generalise. Normalisation feels like a preprocessing detail, something you sprinkle on before a diff. It isn't. Every rule you add or omit is you deciding what "changed" means, and if you never decide, your comparison will happily measure something you didn't intend — in my case, the platform's syntax highlighter.

When you can't get the proof, remove the target instead

I run a second mirror to a different platform, and that one has a native git integration — I push, it publishes. No API calls, so no comparison, so no no-op log. The canary-zero trick simply isn't available.

Rather than pretend it was fine, I changed what the initialisation protected: connect the integration with zero articles in the repository, then add them back one at a time.

If there's nothing to overwrite, the first run can't overwrite anything. I couldn't prove the machinery was safe, so I made the blast radius empty and grew it deliberately — one article, then another, then a few.

Same goal, different mechanism, because the mechanisms have different properties. I think that's the more useful takeaway than either trick on its own: decide what the initialisation has to guarantee, then pick the technique that can actually deliver it here.

The other edge of the same knife

One honest postscript, because a sync this decisive cuts both ways.

It sends whole articles, which means the publish flag in my repo is authoritative too. I once pushed a stale local copy of a published post that still said published: false, and the sync faithfully unpublished a live article.

That's not a separate bug. It's the same property that makes the design work — the repo is the truth and the platform is made to match it — applied to an input I hadn't checked. The more self-healing your sync is, the more precisely it will reproduce a mistake in its input. Mine now has a documented pre-push check for exactly this.

Takeaways

Initialise from the live system, so the first run is a no-op by construction. Then treat that no-op as your proof that the comparison is correct — it's the only test you get to run before the writes start.

Write the dry-run mode first. Idempotence demonstrated after your first production write is a postmortem, not a test.

Treat normalisation as the definition of "different," not as cleanup. Ask what the platform silently changes about your content, because whatever that is will look like an edit forever.

And if the proof isn't available for your mechanism, shrink what can be damaged until the first run is safe by arithmetic instead.

What does your deploy target quietly rewrite after you hand it your content?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

My Fact-Check Caught the Viral Claim and Missed My Own Notes

HideyukiMORI — Tue, 04 Aug 2026 09:44:25 +0000

I thought I had verified it. Six days later I found out I hadn't — not all of it.

A post crossed my feed claiming that a very high share of engineers at one AI company run self-improving agent loops. I don't repeat numbers I haven't sourced, so before reacting I went looking for where it came from. That wasn't caution for its own sake: if the claim were true, I'd have had to rebuild how I run my own agent fleet.

I couldn't find that sentence in the public record.

What I actually found

Three separate, real things — none of which was the claim.

An engineer there is reported as saying he no longer writes code by hand, with a record day of about 150 pull requests. I only ever reached secondhand accounts of that, so I'm passing it along as a report, not as a measurement.

Anthropic's leadership has publicly estimated that 90% or more of their code is written by Claude, including scripts and experimental code. That's an estimate, and the source says so.

And there's a published measurement: as of May 2026, more than 80% of the code merged into their codebase was authored by Claude.

Every source described how much code the model writes. None described how many humans run the loop.

The viral version had swapped the subject — code became engineers — and borrowed "90%" from a different sentence about a different thing. Nobody had to lie for that to happen. A number just drifted one noun to the left.

Absence is the hard part

I want to be careful about what I'm claiming, because this is where fact-checks usually overreach.

I searched from several angles, pulled the top sources, and had each claim challenged by verifiers whose job was to refute it rather than confirm it. The verdict was unanimous. But unanimous about what? Not that the sentence doesn't exist. Only that I couldn't reach it.

I've been burned by this exact gap before. I once read a grep returning zero hits as proof that something wasn't there, and it was — my pattern was wrong. I've also had a search API's "no results" turn out to be a display cutoff rather than an empty set. Zero hits means your search stopped, not that the world is empty.

So the sentence I'll defend is "I couldn't find it in the public record," and not "it does not exist." Those are different claims, and only one of them is something I can actually support.

The search did hand me something useful, though. A published analysis of about 400,000 Claude Code sessions from roughly 235,000 people, between October 2025 and April 2026, found that people make about 70% of the planning decisions but only about 20% of the execution ones. I keep a human gate on every go/no-go call in my setup, and I'd half-assumed that made me slow to automate. Turns out it's roughly where the measured division of labour already sits.

Then my own note failed the same check

Six days later I was about to post a short version of all this. My rule is to re-fetch primary sources before anything with a number in it goes out, even when the research was already verified — days passing counts as drift.

Four claims in the draft. Three held. One didn't.

My research notes had recorded something like "only 0–20% of tasks can be fully delegated." That phrasing appears nowhere in the study. There's no "delegable" anywhere in it, and no 0–20% range. The most likely explanation is that I read "humans make about 20% of execution decisions" and quietly rewrote it as "0–20% of work can be delegated" — which is a different statement about a different thing.

Which is to say: the same swap I had just caught in someone else's number, in my own summary, six days later.

The check I built for other people's numbers never ran on my own.

What caught it wasn't the elaborate part. Not the multi-angle search, not the adversarial verifiers. It was one plain re-fetch of the source document, at the last possible moment, because a rule said to.

The gap wasn't verification. It was the return path.

Here's the part that actually changed how I work.

When I fixed that line, I fixed it in the outgoing post. The research note that produced the post stayed wrong. It sat there for six days, untouched since the day it was written, with no correction task filed against it — and I only noticed because I went back to write this article and checked.

My published work had a gate. The notes that fed it did not. Corrections travelled outward to the artifact and stopped, when the thing that needed fixing was upstream of everything the artifact would ever become.

That's fixed now, and I'd rather show it than assert it. The note carries a strikethrough on the wrong figure, a dated correction marker with the right one, and a line saying how it was caught — because silently editing a note leaves no evidence that anything was ever wrong. A second person re-fetched the source independently and confirmed the original text contains neither the word nor the range. And the missing step got written down as a rule: when you correct a number, immediately grep the whole workspace for the same claim and kill every copy.

Takeaways

For any number-shaped claim, confirm the subject and the denominator in a primary source before you let it move your roadmap. Most viral statistics aren't fabricated — they're real numbers describing something adjacent.

"I couldn't find it" and "it doesn't exist" are different sentences, and only one of them is verifiable. Say the one you can support.

Run your verification on your own notes, not just on other people's claims — and make sure the correction travels back to where the note lives, not just to what you published.

Where do your corrections go: into the post, or back into the note?

Sources: When AI builds itself (the >80% measurement and the 90%+ leadership estimate) · How Claude Code is used in practice (the ~400,000-session analysis and the 70%/20% split)

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

My Upload Check Trusted the Attacker's Word: 8 Bytes Fixed It

HideyukiMORI — Fri, 31 Jul 2026 15:12:50 +0000

A QA pass sent my document uploader a Windows executable that declared itself application/pdf. It was accepted, stored, and listed like any other document.

The MIME allowlist that was supposed to stop it had run, matched, and passed. It wasn't buggy. It was reading a value the uploader had written.

This is the companion to my Bug Smash post about environment variables poisoning a test run — that one was about my machine lying to me. This one is about my input lying to me.

What the check actually covered

The uploader had what looked like several layers of protection. Laid out by who controls each value, the picture changes:

Layer	What it checked	Who controls it	Stops a spoof?
`accept=".pdf,.jpg,.jpeg,.png"`	file picker filter	the browser UI	❌ cosmetic only
Declared MIME allowlist	a string in the multipart body	the uploader	❌
File extension	the filename	the uploader	❌
First bytes of the payload	the actual content	the file itself	✅
`nosniff` + `attachment` on download	how the browser treats the response	the server	⚠️ mitigation, not a fix

Three of those layers were reading values the uploading client had chosen. The allowlist contents were correct — PDF, JPEG, PNG, exactly what the compliance rule says. It just never saw the file.

I wasn't validating the file. I was validating a string the uploader chose.

That bottom row deserves a note, because it's the one I could have hidden behind. The download path already sent X-Content-Type-Options: nosniff and Content-Disposition: attachment before any of this happened. That mitigation is real, and it predates the bug — which is exactly why it isn't a fix. It changes what a browser does with a file that I already agreed to store. Rejecting at intake is a different question, and I hadn't answered it.

How it was found

Not by a scanner. By a person following a QA script, escalating one step at a time: send a .exe (rejected, good), send an .svg (rejected, good), then send the .exe bytes with a spoofed application/pdf content type.

That third step is the one automated checks tend not to reach, because it requires deciding to lie about your own request.

The fix: read eight bytes

Keep the declared-MIME allowlist as a cheap first gate, then put content sniffing behind it. No new dependencies — no finfo, no library, composer.json untouched:

private function sniffMimeType(string $tmpPath): ?string
{
    $handle = @fopen($tmpPath, 'rb');
    if ($handle === false) {
        return null;
    }

    $header = fread($handle, 8);
    fclose($handle);

    if ($header === false || $header === '') {
        return null;
    }

    if (str_starts_with($header, '%PDF-')) {
        return 'application/pdf';
    }
    if (str_starts_with($header, "\xFF\xD8\xFF")) {
        return 'image/jpeg';
    }
    if (str_starts_with($header, "\x89PNG\r\n\x1A\n")) {
        return 'image/png';
    }

    return null;
}

Eight bytes because the PNG signature is eight bytes long; the other two are shorter prefixes of the same read. The call site is four lines:

$sniffed = $this->sniffMimeType($input->tmpPath);
if ($sniffed === null || !in_array($sniffed, self::ALLOWED_MIME_TYPES, true)) {
    throw new MimeTypeNotAllowedException($input->mimeType, $sniffed ?? 'unknown');
}

Two decisions in there matter more than the signatures.

Undecidable means rejected. An unreadable file, an empty file, or anything whose first bytes don't match a known signature all return null, and null fails. A sniffer that returns "unknown" and then shrugs is not a gate.

The two rejections say different things. A disallowed declaration gets File type 'x' is not allowed. Only PDF, JPEG, and PNG are accepted. A spoof gets Declared file type 'application/pdf' does not match the file content ('unknown'). Only genuine PDF, JPEG, and PNG files are accepted. Collapsing those into one message would have been less code and would have thrown away the only signal that distinguishes a confused user from someone probing you.

One thing I did not do: SVG is rejected, not sanitized. An SVG renamed and re-declared as PNG fails the signature check and never reaches storage. If you need to accept SVG, this post doesn't help you — that's a different problem with a much larger surface.

The test that couldn't be written

Here's the part that changed how I read my own test suites.

Before the fix, the test helper passed tmpPath: '/tmp/fake-upload'. That's a string. There was no file at that path. There had never been a file at that path.

So a content-based test wasn't merely missing from the suite. It was impossible to write. The validation logic stopped at the declared value because the fixtures had no content for it to go on to.

The check was as deep as my fixtures were.

The line that unlocked everything was tempnam() — create a real temporary file, write real bytes into it, delete it in tearDown(). Once fixtures were files instead of strings, the new cases wrote themselves: a spoofed .exe asserting the mismatch message, an SVG declared as PNG, and genuine JPEG and PNG regressions to prove the gate still lets real documents through.

What's still open

The vulnerability is closed at intake, and I'd rather end on the part I haven't finished than on the part I have.

The end-to-end test that originally caught this still carries its discovery-time assertion:

expect(listed > 0 || modalText.length > 0, 'upload resolved (accepted or errored)').toBeTruthy();

Read that carefully. It passes when the spoofed file is accepted and listed, and it passes when the upload errors out. It was written to observe what happened, back when nobody knew. It would go green whether or not my fix works.

I closed the vulnerability and left the test that found it unable to prove it. A test written to demonstrate a bug does not become a regression test until you flip its expectation.

What I'd tell past me

If the value you're validating arrived in the request body, you're validating the attacker's input, not the file.
Undecidable must mean rejected. "Unknown, so I'll allow it" is not a gate.
Fixtures set the ceiling on what your tests can check. A fixture that isn't a file can never catch a content bug.

What's still in your upload path that the uploader gets to declare?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

Fact-Checking My Own Blog Posts Turned Into Product QA: 3 Real Bugs

HideyukiMORI — Fri, 31 Jul 2026 15:08:02 +0000

I was fact-checking a sentence in one of my own blog posts — "the API returns no-store" — and found out the API did no such thing.

So I didn't edit the sentence. I fixed the product.

I have a rule for myself: before I publish anything technical, I go through it and check every factual claim against the actual code. Not to impress the reader — to keep myself honest, because it's embarrassingly easy to write something that sounds true. This week that habit stopped being about the writing. Reviewing four posts turned up three real bugs, in the product, not the posts.

Bug 1: the sentence was right, the product was late

The post was about an auth-header quirk on shared hosting, and it claimed the API returned Cache-Control: no-store on authenticated responses. Reasonable thing to claim. Also, when I actually checked, not true — the header wasn't being set.

Because of how the app worked around a proxy that strips the standard Authorization header, the usual "responses tied to Authorization aren't shared-cacheable" protection didn't apply. Which meant an authenticated response could, in theory, land in a shared cache.

I had two options: soften the blog sentence, or make it true. I added a small middleware that sets no-store at the outermost layer of the pipeline, left a couple of tests behind, and shipped it. The sentence is now correct because the product changed to match it — not the other way around.

That's the case I like most. My writing was ahead of my code, and checking the writing is what surfaced it.

Bug 2: "nothing accumulates" — except the thing that did

Another post described a disposable-demo setup: click a link, get a throwaway tenant, and an hourly sweep deletes everything, so nothing piles up. I went to verify the "nothing piles up" claim literally.

Nearly nothing piled up. But each demo tenant left behind a small per-tenant stamp file — a throttle bookkeeping artifact — that the sweep wasn't cleaning. Every demo click added one and never removed it. Not a leak that would take the server down soon, but a direct contradiction of a sentence I was about to publish.

The sweep now removes those stamps when it deletes a tenant, and self-heals any orphans from tenants that are already gone. Claim restored to true.

Bug 3: the demo was showing the future

While writing the English version of that demo post, I clicked through the seeded data the way a reader would. The demo plants realistic history — a few invoices, some payments — so it looks like a live business.

Open it in the first half of a month and some of that "history" was dated in the future: a payment recorded for a date that hadn't happened yet, a paid invoice due next week. The seed used fixed day-of-month values, so early in the month they landed ahead of today.

The person most likely to notice that is an accountant — exactly the audience the demo is for. The fix clamps every seeded event date to "today or earlier," while leaving genuinely future dates (like due dates) alone. I added a test that seeds on the first of the month — the worst case — and confirmed it failed before the fix.

Being honest about the yield

Four posts, three product fixes — but I don't want to oversell the hit rate. Two of the four posts had zero product bugs; their claims were already true, and checking them just cost me an hour and bought me confidence. The three bugs clustered in two of the posts, not one per article. If I dressed this up as "every blog post hides a bug," that would be its own small fabrication.

The point isn't a reliable yield. It's the lens.

Why prose catches what tests miss

Tests check the assertions you thought to write. Prose makes a different kind of claim, and a more dangerous one: "the system does X," stated plainly enough for a stranger to falsify.

Your code doesn't say no-store. Your blog post does. Your test suite never asserted "nothing accumulates" or "no dates are in the future" — but your paragraph did, out loud, to the whole internet. Writing forces you to compress behavior into flat declarative sentences, and flat declarative sentences are exactly the thing you can walk over to the code and check.

So publishing under a rule of "every claim gets verified against the code" isn't just hygiene for the post. It's a QA pass driven by a question your tests never asked: is what I'm telling people actually true right now?

And when the answer is "no, but it should be" — like the no-store case — the honest fix isn't to edit the sentence down. It's to make the sentence true.

Do you check your posts against the code before you hit publish — and has it ever turned up a bug in the thing you were writing about?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

My Generator Hid a Lint Error: CI Never Checks What Nobody Commits

HideyukiMORI — Thu, 30 Jul 2026 15:49:46 +0000

The lint error lived in a file my CI had never seen. It couldn't have seen it — the file only exists after you run a generator with a particular flag, and nobody had ever committed that flag's output.

The setup

NENE2, my framework for typed business apps, ships frontend scaffolding generators: gen:entity builds the typed API layer for a resource, gen:feature builds a state-machine-shaped feature on top. They're template-based and deterministic on purpose — deterministic output is what makes generated code reviewable and lets AI agents use the generators safely.

And like most generators, they have flags. gen:entity <noun> --write adds mutation hooks and write handlers on top of the read-only default. Flags mean branches. Branches mean output shapes that may exist nowhere in your repo.

That's where the bug was.

The bug

The entity mutations template emitted this:

UseMutationResult<Noun, AppError, unknown>

The third type argument, unknown, is exactly the parameter's default — so @typescript-eslint/no-unnecessary-type-arguments flags it. Which means every single output of gen:entity <noun> --write failed lint, in a codebase where CI runs ESLint with --max-warnings 0.

And CI was green the whole time. The generator's determinism tests passed — they verify the templates produce stable, expected output. The committed exemplar for the default archetype passed lint, because it's real committed code and CI lints it like everything else. But the --write branch's output had never been committed anywhere, so no type-checker and no linter had ever run over a single line of it.

A determinism test proves your generator is consistent. It will happily prove it's consistently wrong.

This is a cousin of a broader failure mode I keep running into: gates that glow green while checking nothing in the branch that matters.

How it surfaced

Not by auditing. By building the next feature.

In PR #1580 I was adding a --mutation archetype to gen:feature — a four-state union (idle | submitting | error | success) for form-style features. That archetype consumes what gen:entity --write produces, so for once, verifying the new thing forced me to actually generate the old thing.

The verification was a full rehearsal on real output: run gen:entity payment and gen:feature submit-payment payment --mutation, then put the generated files through the same gauntlet as committed code — type-check, ESLint with --max-warnings 0, Prettier, and the generated transition tests against a real mock server.

Lint went red on code no human had written and no CI had judged. There it was.

The fix

One line in the template. Drop the third type argument — TVariables defaults to unknown anyway, so the emitted types are identical and behavior doesn't change. The PR body records the before/after; the generated output now comes out type-check clean, lint clean at --max-warnings 0, with its four transition tests passing (4/4, exercised against mocked 200 and 500 responses).

The fix took a minute. The bug had unlimited shelf life. That asymmetry is the whole story.

The general lesson: enumerate your generator's branches

If your codebase has a generator with flags or optional archetypes, each branch of its output is code you're shipping to your future self — and CI's default posture toward it is total blindness. Two ways to close the gap:

Keep a committed exemplar per branch. Generate one real instance of every archetype/flag combination and commit it as living code. CI then type-checks, lints, and tests it forever, for free. Drift between templates and exemplar shows up as a normal red build.
Add a generate-and-check smoke to CI. Generate every branch into a temp dir, run type-check + lint + tests over the output, throw it away. No repo noise; costs CI minutes instead.

Either works. What doesn't work is what I had: determinism tests that hold the generator's output stable while nothing ever asks whether that output is valid.

Full honesty about where I actually am: PR #1580 did option 2 manually, once — a rehearsal, not a gate. The PR body itself flags a committed exemplar for the mutation branch as a follow-up. So the branch that bit me is verified today and unguarded tomorrow, and I'm writing this partly so I can't quietly forget that.

Takeaways

Every generator flag is a code branch CI has never met. Green CI says nothing about output nobody commits.
Determinism/golden tests check stability, not validity — pair them with a committed exemplar per branch, or a generate → type-check → lint → test smoke.
The cheapest audit is a rehearsal: generate each archetype for real and run your normal check over the output. My first rehearsal of an old flag found a lint error of unknown age.

Primary sources

NENE2 PR #1580 (merge c2b5311) — the --mutation archetype, the rehearsal, and the one-line template fix

If you have a generator in your codebase: when did the output of every one of its flags last pass your linter? I'd honestly love to know if anyone gates this properly.

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

My Committed Bundle Went Stale: CI Rebuilt It and Never Compared

HideyukiMORI — Thu, 30 Jul 2026 15:38:47 +0000

I found out that a bundle committed to one of my repos had gone stale because an unrelated PR left my git tree dirty.

Not a test. Not a review. A dirty tree, noticed by accident.

The setup

nene-concierge is one of my self-hosted business tools. Its repo commits a built frontend artifact: public_html/admin/app.js, the admin bundle, produced by esbuild via npm run build. The public_html/ tree is the deployable root, generated files included.

Committing build output is a choice with a known tradeoff. I knew the tradeoff. I just wasn't paying for it.

What broke

An earlier refactor (#178/#179) added "type": "module" to frontend/package.json.

That one line changed esbuild's CJS interop output. Every helper call in the bundle went from __toESM(x) to __toESM(x, 1). From that day on, the app.js sitting in main no longer matched what a clean build produces — a 573-insertion diff of pure drift.

Nothing failed. Type-check green. Lint green. Tests green. Nothing in the pipeline had any opinion about that file.

Here's the detail that stings most. My CI runs npm run check, and check includes build. So CI was rebuilding that bundle on every single run — producing the correct, fresh output in its own workspace — and then throwing it away without ever comparing it to the file committed two directories over.

A committed artifact is a claim about your source code, and nothing was checking the claim.

How it surfaced

By luck. While verifying an unrelated PR (#183, a lint-suppressions ratchet gate), I ran the full check locally and git status came back dirty: npm run build had rewritten public_html/admin/app.js under me.

That PR's body records it as an explicitly out-of-scope finding, the artifact was restored so the ratchet PR stayed clean, and the drift got its own issue: #185.

If that ratchet PR hadn't happened to run a build on my machine, I have no idea when I'd have noticed.

The fix — and what the fix doesn't fix

PR #186 (merged as b7eb1ea) is deliberately boring: run npm run build, commit the output as-is. Machine diff only, zero source changes, two generated files touched (app.js and its sourcemap).

Two things made me trust it:

Idempotency: two independent builds, byte-identical output. If your bundler embeds timestamps or random chunk names, a freshness check is off the table until that's fixed — so prove determinism first.
Clean tree after: rebuild, then git status shows nothing. That was the acceptance criterion on the issue.

But notice what this fix is: a resync. It repairs this drift and does nothing about the next one. Any future change that shifts compiler output — a bundler upgrade, a tsconfig flag, another package.json line — re-creates the exact same silent lie.

The actual lesson: freshness must be a test

If a build artifact lives in git, CI has to prove it's fresh. The check is about four lines:

- run: npm ci && npm run build
  working-directory: frontend
- name: Committed artifacts must match a clean build
  run: |
    test -z "$(git status --porcelain)" || {
      git status; git diff | head -50
      echo "::error::Committed build output is stale. Run npm run build and commit."
      exit 1
    }

Build from a clean checkout, then assert the tree is still clean. That's it. It would have turned this whole story into a red X on the PR that added "type": "module", instead of a discovery-by-accident later.

Full honesty: as I write this, that gate is not yet in nene-concierge's workflow — the resync is merged, the guard is the follow-up. I checked the workflow file before writing this paragraph, because claiming a fix you haven't shipped is exactly the kind of lie this article is about.

And the cleaner alternative deserves saying out loud: if you can avoid committing artifacts at all — build in CI, build at deploy — do that. An artifact that's never committed can't go stale in the repo. Committing output is sometimes the pragmatic choice when the deploy target is a plain host that just serves files; that convenience is fine, but it isn't free. The freshness check is the price.

Takeaways

A committed artifact is an unverified claim. CI must rebuild and git status --porcelain-assert it, or the artifact will eventually lie.
Toolchain config changes ("type": "module", bundler bumps) can change output while every semantic check stays green — drift needs no bug to happen.
Prove build determinism (two builds, byte-identical) before you rely on any freshness comparison.

Primary sources

nene-concierge issue #185 and PR #186 (merge b7eb1ea) — the drift and the resync
nene-concierge PR #183 — the out-of-scope note where the drift was first recorded

Do you commit build output anywhere — and if so, what would actually fail today if it went stale? If the answer is "nothing," I'd check.

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

Half My Tests Failed and None Were Broken: My Shell Poisoned PHPUnit

HideyukiMORI — Wed, 29 Jul 2026 15:00:34 +0000

For about a day I believed I had roughly 150 broken tests. I was wrong — not one of them was broken. Then I was wrong about the cause. Twice.

Here's the walk from "our test suite is rotting" to "one stray line in my shell was winning a fight I didn't know it was in."

The symptom

I ran the suite and about half of it was red. Every failure was the same shape: 403 org-access-denied. Tenant-scoped tests, denied across the board.

It looked exactly like accumulated test debt — the kind of thing you file, sigh at, and schedule for "later." So that's what I did. I opened an issue: ~150 failing tests, org access denied.

Correction #1: the tests were fine, my machine wasn't

Before scheduling the cleanup, I ran the suite in a clean environment — a fresh shell with none of my profile loaded.

All green.

Not "fewer failures." Zero. The tests were fine. The failures were a property of my environment, not the code. Which is a worse feeling than a broken test, honestly, because it means the call is coming from inside the house.

The mechanism turned out to be a PHPUnit detail I'd never had to think about: env vars you set in the config do not override an env var that's already set in your shell — not unless you mark them force="true". My shell profile exported a variable that the test config also set, and by default, the shell won. So my tests were quietly running against the wrong context, and everything tenant-scoped got denied.

Correction #2: I blamed the scary variable, not the guilty one

Here's the part I'm least proud of and find most useful.

My first theory for which variable was a JWT signing secret. It's the dangerous-sounding one; of course a bad secret breaks auth. I wrote it up that way.

It was wrong, and the code says why. A signing secret is symmetric: if signing and verifying both read the same (wrong) value, the tokens are still internally valid. A polluted secret doesn't produce 403 org-access-denied — it produces valid tokens for a wrong-but-consistent world. It couldn't be the cause.

The actual culprit was boring: a tenant-slug variable. My shell had it set to one thing; the tests minted tokens for another. So the request would resolve one tenant from the env, carry a token for a different tenant, and the access check — correctly — denied it. Every tenant-scoped test, 403.

I'd spent my first guess on the variable that sounded like a security problem, when a plain identifier was doing all the damage.

The fix was one attribute

force="true" on the env entries, so the test configuration wins over whatever the shell happens to export. That's it. The suite is now hermetic: it runs the same on my machine, on a colleague's, and in CI, regardless of anyone's shell profile.

I proved it the honest way — by injecting the bad variable on purpose. With the injection and no force, the suite collapses (about 150 failures and a pile of errors). With force, the same injection does nothing; everything stays green. The fix is verified against the attack, not against my memory.

What I can't tell you

I can't tell you the exact line that was in my shell profile that day. I don't have it pinned down, and I'm not going to invent it to make the story cleaner. What I have is a reproduction — inject the variable, watch it break — and that's what the fix is proven against.

The count is fuzzy too. It was around 150 failures out of somewhere near 290 tests, and both numbers drifted day to day as the suite changed. "156 out of 291" would look precise and be a small lie. Half the suite red, zero tests actually broken is the true and useful shape of it.

Two takeaways

Make your tests hermetic. Test configuration should beat the machine it runs on, unconditionally — force, or the equivalent in your stack. A suite whose result depends on the developer's shell isn't testing your code; it's testing your dotfiles.

When you're debugging, suspect the boring variable. I lost real time pointing at a secret because it sounded dangerous, while a plain tenant id sat there quietly denying everything. The scary-looking cause is a great way to feel productive and stay wrong.

What's the worst "it's the tests" you've had that turned out to be your own environment?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.

Pilot Then Fan Out: Killing Unknown Blockers in 2 Repos First

HideyukiMORI — Wed, 29 Jul 2026 14:59:58 +0000

I maintain about a dozen small products that share one homemade framework. Last week I built a conformance linter — a little tool that scans each repo for architectural drift — and I was one command away from wiring it into all of them at once.

I didn't. I ran it in two repos first.

That two-repo dry run is the cheapest safety I've added all year, and it caught two things that would otherwise have gone off in every repo at the same time. One of them was the linter being wrong about my own code.

Why fan-out is tempting, and why it bites

When you have a change that clearly belongs everywhere — a lint rule, a CI step, a dependency bump — the obvious move is to apply it everywhere. One sweep, done.

The problem is that a shared tool doesn't fail in the tool. It fails in the consumers, and consumers differ in ways the tool's own tests never exercise. So "all green in the framework repo" tells you almost nothing about what happens when a dozen repos actually pull it in.

If you fan out first and discover the blocker second, you don't get one failure. You get the same failure a dozen times, concurrently, and now you're triaging a wall of red instead of a single problem.

The pilot: two repos, chosen to disagree

The trick isn't "test it somewhere first." It's picking the right somewhere.

I have two ways a repo consumes the shared framework:

one pulls it from a package registry at a pinned version,
the other uses a local path repo — a symlink to a checkout that moves with development.

So the pilot was one of each. Not two similar repos — two that maximize the difference along the axis most likely to break. If a change survives both consumption modes, it'll probably survive the rest. If it breaks, it breaks here, cheaply, where I'm paying attention.

Blocker 1: the local green was a lie

The linter ran fine on my machine in every repo. Then I pushed it to the two pilots' CI and both went red immediately — a hard fatal, exit 255, before the linter did any work.

The cause: the linter loaded the framework's own dependencies to run. On my machine that resolved, because my local setup symlinks to a full checkout of the framework, dependencies and all. But a consumer doesn't have the framework's private vendor/ tree, and CI does a shallow clone without installing it. So the exact same command that was green locally was a guaranteed fatal in every consumer.

"Works on my machine" had a specific, mechanical reason to lie: the symlink was hiding the fact that consumers don't get the tool's dependencies. The fix was to resolve against the consumer's dependency tree instead of the tool's. Small change. But if I'd fanned out first, it would have been a dozen red pipelines at once, all with the same confusing exit 255.

Blocker 2: the linter was wrong about my own fleet

The second one was more humbling.

One of the linter's rules flags hardcoded default secrets — a real thing you want to catch. But my correct, intended pattern for dev secrets routes them through a guarded resolver that refuses to use them in production. The rule saw the literal and screamed, not understanding that this literal was the safe, guarded case.

So the linter's very first real run produced a false positive in roughly eleven of my products at once — every repo that used the guarded pattern, which is to say every repo doing it right.

That's the nightmare version of a fleet-wide rollout: a tool that's confidently, uniformly wrong. If those false alarms had landed in a dozen repos on the same afternoon, the rational response would have been to distrust the linter and turn it off — killing the whole effort. Instead, the pilot showed me the rule needed an exception for the guarded pattern before anyone else saw a single false alarm.

What I'd have lost by skipping the pilot

I want to be honest about this part: the "dozen simultaneous red pipelines" is a thing that didn't happen, so there's no log of it. It's a projection, not an incident.

But it's a well-supported projection. Both blockers came from structure shared by every consumer — the dependency layout and the guarded pattern — so both would have fired everywhere. The pilot didn't get lucky with two repos; it exercised the two conditions that made the failures universal. That's the difference between "I tested it" and "I tested the thing that varies."

The method, generalized

Unknown blockers live in the consumer, not the tool. A shared tool's own green suite doesn't cover the environments it will run in. Assume the interesting failures are downstream.
Blast radius = consumers × shared structure. If a failure comes from something all consumers share, fanning out multiplies it. Sequence it instead.
Pick the pilot to span the variation axis. Two repos that differ where breakage is likely beat ten repos that are all the same. Diversity, not count.
Leave a stop valve. Every rollout step — human or agent — should be allowed to halt and escalate when a premise looks wrong, instead of pushing through.

There was a bonus, too. Once the linter was correct and did fan out, it immediately surfaced a genuine latent auth gap in one repo that everyone had walked past. The tool I built to prevent drift turned out to be a decent detector of the drift already there. But it only earned that trust because it didn't cry wolf a dozen times on day one.

When you roll a change across many repos, what's your pilot — and how do you choose which repos go first?

── Hideyuki Mori (Ayane International) 🔗 hideyuki-mori.com

HideyukiMORIFollow

Building API-first PHP tools for self-hosted business workflows. Creator of NENE2 and the NeNe OSS series.