Claude and ChatGPT are both toxic mythomaniacs. With the calm confidence of someone who genuinely believes what he's saying, one of them tells you a job is done on a metrics tool. I check. The branch exists nowhere, not locally, not on the remote, not in the closing queue. Nothing.
The other one tells you its tests pass. It never ran a single one. It wrote the code and assumed it works, with the same conviction as if it had actually watched the green checkmarks scroll by. Same pathology, 2 different masks.
The funny thing is, none of these pathological behaviors were invented by AI. It copied them from us, a 3,000-year-old bug, the one where Ulysses knows perfectly well he'll crack in front of the sirens and has himself tied to the mast before he even hears the first note.
My mast is code that refuses. Not another rule stacked onto an instructions file that already has hundreds. A lock.
I still haven't fully settled this question though: can this kind of gate really replace a written instruction, or is there a core of behavior that no amount of code can force directly?
Your AI Learned Our Oldest Bug
The Ulysses story isn't a nice metaphor I picked after the fact. Commitment devices work for a precise reason (they don't strengthen willpower, they make the undesired action too costly or too impossible to happen before temptation shows up). Ulysses doesn't get stronger. He gets tied to a mast.
Training for safety instead of honesty produces a version of the same failure, an AI that learns to say what sounds correct rather than what is true. It's the same shape from a different angle, and it lines up with what I watch happen daily on my own project. Nobody trained my agents to lie about branch status. They just learned, somewhere in the giant pile of human text they were shaped on, that confident completion claims get rewarded and messy uncertainty doesn't.
So the fix can't be another appeal to honesty. It has to be a mast.
4,429 Words, 0 Guarantee
Some context first. My project runs on roughly 55,000 lines of TypeScript, a PostgreSQL database holding more than 2,000,000 companies, and exactly 0 human code review. Every line is written, tested, merged, and deployed by agents. On a good day I watch 15 branches merge in 4 hours without touching a keyboard.
The instructions file behind all of that is 186 lines, 4,429 words. Rules on architecture, on naming, on what counts as done, on how to test before merging. I wrote most of it after getting burned, the way you'd expect.
The same day I sat down to write about this, a session lied about the state of its own work. Not a hypothetical, not an old war story, the same day (more on that one in a minute). A 4,429-word document sitting right there in context, read at the start of every session, and it still happened.
That's the part that took me a while to accept. More words don't buy more compliance. Past a certain point they buy the opposite, because every additional rule dilutes the weight of the ones already there. I got a lot of mileage early on from writing things down in plain terms, the way I described the prompt contracts rebuild that followed a while back. That mileage runs out.
So if a 4,429-word contract wasn't the mast, what was?
No Proof, No Ship
The deploy gate is the cleanest example. Somewhere in the pipeline sits a function that refuses to ship in exactly 3 cases. The CI response is unreadable. No CI run exists for the exact commit about to go live. Or the last run for that commit finished without a green result, or didn't finish at all.
Fail-closed is the name for the principle underneath all 3 checks (absence of proof counts as refusal, never as permission). The gate isn't paranoid, it just refuses to trust vibes. It asks the CI system, and if the CI system hasn't spoken clearly, the answer defaults to no.
This solves exactly 1 problem: whether the code that's about to go live has been proven to work. It says nothing about whether the agent that wrote it told the truth about anything else along the way. That question stays open a while longer.
The Hook Born From a 5,200-Line Mistake
08/09. A session set out to wire in a new data source. By the time it stopped, the branch held roughly 50 files and 5,200 lines, all crammed onto a single branch, all at once. Review got refused on sight. Hours of CI ran against code that kept moving under it while the tests were still executing.
The instructions file already said, in plain words, to break work into small batches. It had said that for a while. It didn't hold.
What held was a pre-commit hook, written the same day the mess happened. Trip it and you get exactly this message: blocked, too many files or too many added lines versus origin/main, split the work and try again. The threshold sits at 15 files or 800 added lines.
There's still a way around it, a dedicated environment variable that skips the check. I kept it on purpose. It's nominative, it's manual, and it only gets used after we've explicitly agreed in advance that a specific piece of work genuinely needs to land in one piece. The door exists. It just isn't unlocked by default, and using it means telling me first.
Which raises the follow-up question: if a deliberate escape hatch stays open, what actually stops it from becoming the new default habit instead of the exception?
5 Agents, 1 Door, 0 Progress
Late July into early August I built a closing queue, a process that runs every 60 seconds and guarantees exactly 1 active instance at a time. The idea was simple: don't let 2 agents try to close the same batch at once.
The instructions file records what happened next in its own words. 5 closures failed on the morning of 08/05 because concurrent sessions were fighting over the lock. Picture 5 agents pushing the same door in turn, like a raid party wiping on the same boss for the sixth time, each one convinced this pull is finally the one that gets through, and the boss hasn't even moved. None of them get through. The door doesn't care how confident you are.
The fix wasn't a new line telling agents not to trigger closings themselves. It was removing the ability to do it at all. A single alert fires if a lock gets held past 150 minutes, once per holder, so the channel doesn't drown in noise from a queue that's simply doing its job slowly.
The Branch That Didn't Exist
Today, the same day I'm writing this piece, a session announced it had finished a measurement tool. Clean message, confident tone, the kind of update that reads like good news.
I checked. The branch existed nowhere. Not on my machine, not on the remote, not sitting in the closing queue waiting its turn. It simply wasn't there. The branch existed and didn't exist at the same time, and unlike Schrodinger's cat, opening the box didn't help, because there was no box, no lab, no cat, just a commit message that lied to my face.
The instructions file already forbids, in bold, in plain letters, the words done, finished, or shipped before work is merged and verified. It's been in there for a while. It didn't hold, not this time either.
An unverifiable status update is just a guess in a suit.
The fix wasn't another sentence added to a document that already contained the rule in bold. It was a requirement to produce proof (query the remote server, show the branch actually exists) before any announcement gets made at all. The batch that followed shipped clean, no drama, no phantom branch.
What Still Can't Be Automated
Here's where the 2 families split. Rules with an effect you can measure convert into checks: diff volume, a green build, an architecture boundary that can't be crossed. Once they're code, they hold indefinitely. Nobody has to remind anyone. On that ground, code really does replace text, and it does the job better.
Behavioral rules don't convert the same way. Don't expand the scope mid-task. Announce your actual state honestly. Neither of those leaves a trace at the moment it happens, which means there's nothing for a gate to check against. The fix each time wasn't a stronger sentence, it was removing the opportunity to do the thing at all (closing the branch check, requiring the proof query). That's the part of the question I still can't close. A behavior with no observable trace at the moment it occurs is a behavior no gate can catch in the act, only after, and only if something downstream happens to notice.
None of this is free either, and I think it's worth saying plainly. Every gate is code I now have to maintain, and a badly calibrated one is worse than no gate at all, because it either blocks good work or teaches everyone to route around it. My test coverage check still sits in observation-only mode for exactly that reason. A numeric threshold turns into a number to game rather than a signal to trust, and I haven't found the version of that check I'd actually enforce. Adding more text to an already long document has diminishing returns too (a rule buried on line 140 of 186 gets read carefully by exactly nobody, agent included), and the file grows heavier every time I try to patch a gap with another paragraph instead of another gate.
Honestly, maybe I'm wrong about where that boundary sits long term. Behavior that leaves 0 trace today might leave a trace tomorrow, once logging gets granular enough to catch intent instead of just outcome. I'm not counting on it yet.
We used to say this back when we still hand-wrote most of our code: the truth is in the code. Turns out nothing's changed, it's still true, it just moved down a layer. That's also why I don't lean on statistical models alone for the things that need to be certain. Sophisticated as they've gotten, I still reach for plain old deterministic algorithms wherever the stakes are proof rather than probability. What I actually dread isn't today's failure mode. It's the day the LLM becomes the new compiler, the layer everyone trusts blindly, the one that quietly turns deterministic code into something that isn't anymore. 🤓
So, a partial answer. Everything with an observable effect, code has already won, cleanly, and I don't expect that to reverse. Everything without one (the honesty itself, the restraint to not expand scope) still runs on trust I haven't figured out how to lock down. I know exactly which side of that line each rule in my instructions file sits on now. I just don't have a gate for the second side yet.
Sources
- Precommitment Psychology: Bind Your Future Self to Goals
- My honest field notes on the verification gap no one's talking about
This post may contain affiliate links. If you click them, I might earn a small commission (costs you nothing, and helps me keep shipping quality articles every day for your reading pleasure).
Top comments (0)