AI drafts, humans sign, tools can only say no

Pin Tsai — Tue, 27 Oct 2026 07:38:51 +0000

Most of the code on our team is written by Claude Code now. Several product lines, one shared database, a small team. Output stopped being the bottleneck a while ago.

Accountability didn't get cheaper, though. It got harder. More changes landed than anyone could read, and the person signing off was signing for things they had never really seen.

This post is about the rules we ended up with, and the small Claude Code plugin that came out of them. I'm mostly writing it because I want people to poke holes in the reasoning.

The first instinct was wrong

When AI-written changes started worrying us, we did the obvious thing: we added more checks. Every AI step got a human gate.

Two things happened:

The AI got slower and more hesitant with each layer.
The humans couldn't keep up.

And when people can't keep up, something quiet happens. They start accepting verbal reports as verification. "It's handled" replaces "show me."

Then we had a payment-related incident. Looking back at the two days before it, the pattern was plain: the decision had run ahead of the evidence, and the gate was a person who was already out of attention.

So the chain broke at the human. But the only one who can answer "should this exist at all?" and "where does it stop?" is also the human. The problem wasn't too many humans or too few. It was human attention spent in the wrong places.

Four rules

1. Humans only make the decisions that matter

What to build. Approve the spec. Approve the code. Write down why. That's the human's whole job. Everything else can be drafted by the AI.

2. The AI questions the human before it codes

Before anything touches money, schema or a shared contract, the AI "grills" the plan. It lays the plan out as a decision tree and asks the human only the questions that belong to a human:

Is the need real? Who named it?
What does it cost to not do it?
Where is the boundary?

Implementation-level questions it answers itself, and for each answer it says where it's most likely to be wrong. The human overturns what they disagree with and signs a one-page sheet. Nothing gets built before the signature.

This flips the usual direction. Normally the human reviews the AI. Here the AI also reviews the human, at the one moment it's cheapest to be wrong: before any code exists.

3. Decisions live in a repo, not in people's heads

Decisions used to live in chat threads, task cards and memory. Three months later nobody could say why something had been decided, only that it had.

Now every signed plan, every override and its reason goes into a separate Git repo we call the brain. Plain Markdown, dated, append-only. The AI reads it before asking anything, so the thicker it gets, the fewer questions climb up to a human.

4. Tools can refuse, never approve

This is the one I care most about.

Hooks, gates, CI and lint get veto power and nothing else. They can block an edit in a red zone. They can refuse a workflow that forgot to pin a cheap model. They can fail a build. They can never say yes.

That's what keeps the AI from quietly becoming the boss: the only way to get a "yes" is a human signature.

Two lessons from building it

A rule that can't physically be pressed isn't a rule. We designed "the human approves the PR on GitHub," and halfway through realised the AI and the reviewer were using the same account. GitHub doesn't let you approve your own PR. The rule looked great on paper and was impossible in practice.

Rules in documents don't hold. Rules in tools do. We had a written guideline about not fanning out to too many subagents on the expensive model. One afternoon a single session spawned sixty-five of them anyway, most on the most expensive model. The guideline moved into a hook that same week.

What it looks like in practice

I packaged this as a Claude Code plugin called Shipright. You mark "red zones" (defaults: **/migrations/**, **/payment/**, **/billing/**). When Claude tries to edit one without a signed decision for the current branch, it gets this:

Blocked: migrations/0001_example.py is a red zone (**/migrations/**).
No consensus doc covers branch feat/example.
Exits: run /shipright:grill, or /shipright:grill --override "<reason>" (recorded in the brain log).

Every block says what was stopped, why, and how to get out. An override is allowed, but it's written down with a reason and a date.

There's one subtlety. If the gate only checks "does a signed doc exist," the AI can just write the doc itself. So the gate has tiers:

Tier	Gate opens when
`file`	A decision doc for this branch exists in the brain
`commit` (default)	The doc is committed and clean in the brain repo, and optionally not committed by the AI's git identity
`github`	The spec PR is approved on GitHub (designed, not built yet)

In commit tier the AI also can't run git commit against the brain from Bash. A human has to.

What it doesn't do (yet)

I'd rather list these than have someone find them:

It watches Edit, Write and NotebookEdit. A Bash sed or echo > into a red-zone file isn't caught.
cd brain && git commit in a single Bash call slips past the commit check.
Hooks fail open on internal errors, on purpose. The only deliberate denies are a broken config and the gate itself.
The "review twice" idea (spec PR, then code PR) is only enforced up to the commit layer today.

Where I'd like your eyes

These are the parts I'm least sure about:

Is "tools only veto" too strict? Some teams auto-merge on green CI and seem fine. Is there a class of change where a tool saying yes is genuinely safe?
Does grilling scale? It works for one human and a handful of risky changes a week. I don't know what happens with ten people.
Is a separate decision repo worth it, versus ADRs inside each product repo? We chose separate because several products share contracts, but it's another thing to keep alive.
How would you close the Bash hole without making the gate so noisy that people turn it off?

If you've built something similar, or tried this and watched it fail, I'd really like to hear how.

The plugin is open source (Apache-2.0): https://github.com/agoopintsai/shipright

DEV Community: Pin Tsai