Prince Panchani

Posted on Sep 17

AI Can Write the Code. Can It Prove the Fix?

#ai #python #opensource #testing

The most expensive thing an autonomous coding agent can produce isn't a broken build. A broken build is free: CI goes red, you move on.

The expensive thing is a green diff that is wrong. It costs the review cycle, the merge, the deploy, and then it costs the afternoon three weeks later when someone bisects it.

And the way an agent produces that artefact is depressingly simple. It writes a change. It runs the tests. One test is inconvenient. It adjusts the test. Everything is green. It says done.

Nothing in that sequence is a lie, exactly. Every step is something a tired human does too. The difference is that when a human does it, another human reads the diff and asks why did this assertion change? When an agent does it at three in the morning across a task queue, nobody asks.

So the question I got interested in isn't "can AI write code?" That's settled. The question is:

What would an AI-generated change have to produce before a human is entitled to trust it?

I found a project trying to answer that concretely, contributed three PRs to it, and got taken apart in review in a way I'm still thinking about. This is what I learned.

Code generation is not verification

Here's the shape of real engineering work:

The middle is not a ceremony. Every stage it skips is one where a specific class of defect is caught.

Consider a ticket: "Fix the crash when a task title contains non-Latin characters." (This is a real class of bug — v0.2.4 of the project I'll describe fixed exactly this on Windows.)

An agent produces a diff. The diff touches an encoding path. Tests pass. Ship?

You can't answer that, because you don't know:

Did a test exist that reproduced the crash? If not, what exactly demonstrates the fix?
Does that test fail on the old code? If it passes on both trees, it proves nothing about the bug.
Did the test count go down? A change that fixes a bug and quietly deletes four unrelated tests is two changes.
Did any assertion become tautological? assert result == result is green forever.
Did an unrelated behaviour change? Encoding fixes love to move line endings.

Notice that none of those is questions about the code. They're questions about the evidence surrounding the code. And an agent that scores its own work has a structural incentive problem: the thing producing the change is also the thing attesting to it.

A plausible diff is not proof. It's a hypothesis with good syntax highlighting.

Enter no_human

no_human is an open-source (MIT), locally-run workflow that takes a ticket through implementation and review and stops at an open pull request. It runs on your machine, on your own Claude credential, against your own checkout. There is no hosted service in the middle of the loop.

Its README describes what it is in one line: from ticket to reviewed pull request. The more interesting framing is what it refuses to do — it never merges. The orchestrator opens a PR and parks at awaiting_approval. Merge is a human command, always.

The project's own documented pipeline:

ticket ──► context ──► plan ──► implement ──► review ──► test ──► PR ──► you merge
              │                      │           │         │
              │                      │           │         └── local runner + optional CI
              │                      │           └── fresh-context reviewer, edit tools refused
              │                      └── Agent SDK, your credentials, your checkout
              └── grep, git log, past sessions

Two things about that diagram are load-bearing and easy to skim past.

Git is not the model's. Branching, committing and pushing are done by no_human's own git code, not by the agent. Implementation runs behind a PreToolUse hook that enforces forbidden paths, protected branches, a merge ban and a destructive-shell circuit breaker. The agent writes files; it does not drive version control.

The reviewer is not the author. It is a separate Agent SDK session with fresh context, a different model tier by default, and the file-edit tools refused.

The gates, and what each one is actually for

This is the part worth reading even if you never install the thing, because each gate exists to defeat a specific failure mode of autonomous work.

Tamper guard — deterministic, no model involved

testing/tamper_guard.py diffs test files separately from product code, and fails on:

a net drop in test count or assertion count
a net increase in skip / xfail markers
a real assertion replaced by a tautology
a behaviour-faking autouse fixture appearing in a conftest.py

No model judgement. It runs before the expensive review, so a self-gutted suite gets blocked before a reviewer token is spent. It covers Python, JS/TS, Java and the e2e/ tree.

Why this matters generally: if your verification mechanism is inside the agent's write surface, it is not a verification mechanism. Any autonomous system that can edit its own tests needs something outside the loop watching the tests themselves.

Reproduction gate — proving the bug existed

testing/repro_gate.py takes the tests the coder offers as evidence, copies them into a worktree at the merge base, and requires them to fail there and pass on the new tree.

reproduction test
  OLD TREE (merge base) → FAIL
  NEW TREE (the change) → PASS

A bugfix whose test also passes on the unfixed code has demonstrated nothing about the bug. This is the single cheapest idea in the project and the one I'd steal for a human team tomorrow.

Accuracy note: the default mode is advisory, which still enforces for a Python bugfix where the coder's edits reached .py files through the edit tools. repro_gate.mode: required drops the conditions and enforces for every change. That nuance is in the docs, and it matters — a .py file written by sed isn't bound by the default.

Independent adversarial review

review/reviewer.py opens a fresh session — it has not seen the coder's reasoning, its rationalisations, or its plan. Its Write, Edit, NotebookEdit and MultiEdit tools are refused, along with direct git and forge write commands. By default it runs on a different, higher model tier than the implementer. It is told to refute "done".

What it returns is the part I care about: a checklist of findings with file, line and severity. A boolean verdict. Never a numeric self-score.

Three properties make that verdict hard to game, and all three are code rather than prompt text:

Every cited location is checked against the actual tree; a finding citing a line that doesn't exist is demoted to advisory.
The pass/fail is recomputed deterministically from the checklist, not taken on the model's word.
A reviewer that crashes, times out, or emits nothing parseable fails closed.

That last one is the tell. A system that fails open on a broken reviewer doesn't have a reviewer; it has a formality.

Held-out tests, verifiers, and merge policy

tests/held_out/ are run by the orchestrator and handed to the reviewer as evidence the implementer never saw.
.no_human/verifiers.yaml lets you write project rules as plain-English statements scoped by path glob; each is put to an independent bounded judge call, and every verdict is recorded — pass or fail, never only the failures. A judge that reaches no verdict after one retry is recorded as unavailable and treated as advisory, on the explicit reasoning that a gap in the gate is not evidence about the change.
core/merge_policy.py evaluates a fixed rule set (review passed, tests passed, tamper clear, repro passed or not required, verifiers satisfied, CI success or unknown) into a single ready: bool plus a per-rule breakdown. If the diff edits the merge policy file itself, that's flagged and forces ready: false — a coder cannot author its own merge gate.

Honest stopping

The loop is bounded: 3 attempts per loop, 500 turns per attempt, 9 lifetime attempts across resumes. Repeating an identical tool call, or the same error signature twice, trips stuck detection, which resets context instead of stacking corrections on a confused session.

When it runs out, it doesn't invent a plausible diff. It classifies the blocker into one of eleven categories (MISSING_ACCESS, AMBIGUITY, SCOPE_EXPLOSION, IMPOSSIBLE, QUOTA, BUDGET_EXHAUSTED, and five more) and either parks with a wake condition or escalates with a structured report and one specific question.

The docs put it better than I can:

An honest escalation costs a minute to triage. A confident wrong diff costs an hour to review.

The two workflows, side by side

The common shape:

Ticket
  ↓
Agent generates code
  ↓
Agent runs tests  ←──── agent can edit these
  ↓
Agent says "Done" ←──── agent grades itself
  ↓
You review a diff with no evidence attached

Failure modes: deleted tests, new skips, tautologies, a fix that never reproduced the bug, a confident answer where "I don't know" was correct.

The no_human shape:

Ticket
  ↓
Context (grep, git log, past sessions)
  ↓
Plan
  ↓
Implement          (git owned by orchestrator, not the model)
  ↓
Tests + held-out tests
  ↓
Tamper guard       ← deterministic, before any reviewer spend
  ↓
Reproduction gate  ← fails at base, passes on head
  ↓
Deterministic evidence (lint, wiring, net-new types)
  ↓
Independent reviewer, fresh context, edit tools refused
  ↓
Merge policy → ready: bool + per-rule breakdown
  ↓
PR opens, task parks at awaiting_approval
  ↓
YOU approve

The agent's opinion of its own work appears nowhere in that chain as a gate.

The part I keep thinking about: confidence vs. evidence

Every agent I've used will tell you it's confident. Some will give you a number.

That number is a model output. It was generated by the same process that generated the code, conditioned on the same context, carrying the same blind spots. Asking it how sure it is is asking the defendant for the verdict.

Compare:

Reproduction test
  OLD TREE → FAIL
  NEW TREE → PASS

Test integrity
  tests: +58   assertions: +0 net loss
  skips: +0    tautologies: 0
  → CLEAN

Reviewer (fresh context, different model, edit tools refused)
  findings: 0 blocking, 2 advisory (file:line cited, verified against tree)
  → PASS

Merge policy
  review ✅  tests ✅  tamper ✅  repro ✅  verifiers ✅  ci ✅
  → ready: true

Every line there is something a second engineer can re-run. None of it requires believing anything the agent said about itself.

Confidence is a claim. Evidence is a claim someone else can check.

That's the whole idea, and it's not really about AI.

Contributing to a project that is itself about trustworthy AI

Here's the recursive part that made this fun: when the project is a verification system, your PR gets verified by the philosophy it implements.

I contributed three changes, all merged, all under issue #114 — "Attach net-new type-checker diagnostics as review evidence; measure before adopting LSP navigation." The premise of that issue is that the review gate runs on machine-checkable signals, and type diagnostics were a missing one.

PR #164 — net-new type diagnostics as review evidence (merged)

The problem. A reviewer looking at a diff has no way to know whether the change introduced type errors. Running a type checker on the head tree is useless: a repo with 400 pre-existing errors reports 400, and the signal drowns.

What I built. If the repo under review configures a checker (pyrightconfig.json / [tool.pyright], [tool.mypy] / mypy.ini / setup.cfg [mypy], or tsconfig.json), the same checker runs with the same argv over the whole project twice — once at the merge base, once at the reviewed commit — and the base result is subtracted. A repo with 400 pre-existing errors reports net-new: 0.

Four design decisions, each avoiding a specific defect rather than adding a feature:

Not scoped to changed lines. Unlike lint evidence. The characteristic net-new type error appears at a call site the diff never touched — narrow a parameter and every caller lights up. A changed-line filter discards exactly the diagnostics worth having.
Fingerprints ignore line numbers. One inserted import shifts everything below it. The key is (path, code, digit-normalised message) as a multiset.
Silence means "did not run", never "clean." A missing binary, a crashed checker, unparseable output — all yield ran=False, which renders no section. A run that happened and found nothing renders net-new: 0, which is a different and usable fact.
Coverage is reported, not assumed. A throwaway worktree carries no installed dependencies, so tsc and pyright degrade unresolved symbols to Any. unresolved_imports renders as a COVERAGE LIMIT line, so net-new: 0 can never read as an unqualified clean.

Review feedback — three blockers, all real. This is where the project taught me something.

The egress gate went red. Every exec or network channel in the repo is declared in tests/test_egress_allowlist.py. My checker built its argv from shutil.which, so the scanner couldn't name the program. The maintainer's point wasn't "add an entry" — it was that the PyPI pyright package is a launcher that downloads Node on first run, so a review could make a network call, and the entry had to say so. Writing it exposed a contradiction in my own module docstring, which claimed "nothing is ever installed."
My checker wrote into the tree under review. mypy drops .mypy_cache/ where it runs; tsc writes *.tsbuildinfo. The collector sits inside the window the orchestrator brackets with reviewer_worktree.snapshot / .compare — so a cache write makes that compare report an added path, the orchestrator charges reviewer_wrote, reverts, and replaces a real verdict with an integrity failure nobody caused. I had reasoned about the collector in isolation instead of about where it sits in the orchestrator. That's exactly where it bit.

The fix was symmetric rather than per-checker: _run_at_commit now serves both sides, so nothing the checker writes can reach the attempt's tree, and untracked files can no longer read as net-new. Pinned by a test using a fake checker that really does create .mypy_cache/missing_stubs where it runs. Reverting the after-run to the reviewed tree turns six tests red.

Cost. One deadline for the whole collection rather than a per-run cap (whose real worst case was checkers × sides × cap), off the event loop via asyncio.to_thread, skipped entirely on the single-turn route.

The thing I'm proudest of is the one nobody asked for. Making the runs symmetric didn't fix the TypeScript coverage problem — it changed its shape. Both sides now degrade equally, so the comparability check passes and the subtraction runs over two blind analyses. Arithmetically sound. But net-new: 0 would have been a false clean, which is the one thing that module exists not to emit. So the COVERAGE LIMIT line stays, and dropping it turns a test red.

And I reported the unflattering number. Detection across 130 local repos: it would render on 32 (25%), of which 94% are the degraded TS path, with a third to two-thirds of what it reports there being unresolved-import noise. I also stated the two limits on that sample — one developer's machine, skewed to TS frontend work, so 25% is not a population figure. It landed as advisory evidence precisely because the coverage line says so out loud.

PR #221 — per-edit type feedback to the coder (merged)

The idea. Phase 1 tells the reviewer. Phase 2 tells the coder, in the same turn that caused the diagnostic. A PostToolUse hook beside the existing lint hook: after an Edit/Write to a .py/.pyi file, run the configured checker on that one file and report what it did not report on that file at its previous successful check. Off by default.

The blocker that mattered most was a sentence, not code. My header said "Your edit to X introduced N diagnostics." The maintainer reproduced two ways it's false:

mypy <file> follows imports. He drove the hook with an app.py that was md5-identical before and after while lib.py gained an error — and got told his edit to app.py introduced it.
Bash isn't in _EDIT_TOOLS, so sed -i, patch, ruff --fix and black are invisible, and whatever they broke surfaces on the next Edit, attributed to that edit.

Neither is a false clean, so neither is the unacceptable outcome. But it's a false signal carrying an imperative, delivered mid-turn, on sequences a real attempt produces constantly. A coder that spends turns fixing something it didn't cause is the exact attempt cost the project is trying to avoid.

The block now reads "N diagnostics on or reachable from X that were not reported when this file was last checked" — naming both ways it can mislead, in the text the coder actually sees.

What that review round taught me: the lint hook can honestly say "your edit to X introduced" because ruff sees one file. A sentence that's true for one component is not automatically true for its neighbour. I had inherited the wording along with the architecture.

Two other things went in that I hadn't been asked for:

A credential scrub. The checker subprocess inherits the process environment, and mypy imports whatever a repo's plugins = line names — so a reviewed repo could run its own code with our OAuth token in the environment. _checker_env drops every secret-shaped variable and keeps PATH, HOME and proxies. It reaches the already-landed Phase 1 too. And because that scrub costs something real (a plugin that reads DATABASE_URL no longer loads), the failure renders a TYPE EVIDENCE: NOT COLLECTED line naming the cause and ending: "This says nothing about whether the diff is type-clean; it says this check did not run."
Cache-dir hardening. One directory per attempt under the system temp root, created 0700, refused unless it's ours alone, reclaimed by owner-pid liveness. os.makedirs(exist_ok=True) follows a symlink to a directory and returns happily; the os.lstat is what catches it. Forcing the predicate true turns five tests red.

Eleven paths in that hook return silence. Exactly one speaks.

PR #229 — measuring before building (merged)

Phase 3 of #114 was supposed to be "add a symbol server so the coder can navigate instead of reading whole files." I read it as a gate, not a feature: measure whether navigation would actually pay before building anything.

So scripts/navigation_value.py measures what share of the coder's file-read mass sits in reads a definition / references / hover call could have answered, net of what the symbol answer would itself cost — against a pre-registered 15% threshold, chosen before the number was known so it couldn't be tuned to the answer.

First run said PROCEED at 27.6%. It was wrong. .md was the largest extension in the corpus by read mass, and no symbol call answers a question about a README. With a closed allowlist of languages a symbol server actually serves, the same corpus reads 10.4% / 13.7% and the verdict inverts. One constant flipped the phase gate, so a fixture pair identical except for the file extension now pins it.

Then the maintainer found the deeper version of the same bug. My SYMBOL_QUERY_TOOLS was {Grep, Search}. no_human's coder never emits either — it searches through Bash. From the fleet database:

Bash 115,776 | Read 35,557 | Edit 18,425 | Write 3,010
Grep 0 | Glob 0 | Search 0
Bash calls containing grep/rg/ag/ack: 50,459

So the class my script called its strongest signal was structurally empty, and the script happily rendered a confident HALT on evidence where its own strongest signal could not exist.

The fix: extract searches out of shell commands per binary (option tables per binary, because -r takes no value in grep and is --replace in ripgrep), and make the zero fail closed — NoSearchChannel refuses a corpus with reads and no searches, because an absent channel is a gap in the instrument, never evidence about the agent.

Against the full fleet database, after the fix: symbol_lookup went from 0 to 15.6% of read mass, and the verdict inverted from HALT to PROCEED. The class that was structurally empty became the largest single contributor.

And the output still carries a caution, because the result got less robust, not more: re-deciding at 6,000 chars gives PROCEED, at 24,000 gives HALT.

What I actually learned

1. Understand the constraints before writing code. Both of #164's hard blockers came from reasoning about my module in isolation instead of about where it sits in the orchestrator's integrity window. The architecture is the constraint.

2. A good PR explains behaviour, not implementation. The bodies that survived review are the ones that said what this establishes and what it does not. The "Not covered" and "What I could not verify locally" sections did more work than the design sections.

3. Report the honest number, not the flattering one. 25% of repos, 94% degraded path, a third to two thirds noise. Saying so out loud is what made it landable as advisory evidence.

4. Tests are part of the product. Every guard I added carries a positive control: red on its own defect, green restored. Twenty-four single-line mutations in #229; twenty-three caught, and the one survivor documented as equivalent code rather than quietly ignored.

5. Measure instead of arguing. Three rounds of hard review, and the finding I disagreed with least was the one I'd have lost by arguing. Re-measuring a number the maintainer gave me — three ways, all agreeing — turned out to matter, because his basis had already moved.

6. More automation is not more reliability. Adding an automated type-evidence signal introduced a way to destroy a real review verdict. Every gate you add is a surface.

7. Human approval is not a bottleneck. It's the trust boundary. It's the one place where responsibility actually lands on a person, and it's cheap compared to everything downstream of a bad merge.

What caught my attention in v0.2.4

Release: https://github.com/no-human-ai/no_human/releases/tag/v0.2.4

Approve-and-merge, fixed — a lesson about where "it works" is measured

In 0.2.1–0.2.3 the packaged desktop app could open pull requests but not land them. The merge gate shelled out through the frozen binary as if it were a Python interpreter, so every nh approve and every board Approve failed at the test step. The gate now resolves a real interpreter and decodes output as UTF-8, end to end.

Why it matters: the human decision point is the product's most important surface. A pipeline that opens PRs but can't land them isn't 90% working; it's broken at the only step that requires a person.

Developer impact: if you're on a packaged 0.2.x build, upgrade before you evaluate the loop — you were evaluating a severed approval path.

The review gate is reusable outside a running server

You can now run it from a session as a plugin skill, or on a pull request in your own repository as a GitHub Action — and a fork's pull request is skipped before any credential is read.

Why it matters: this is the change that most affects teams rather than individuals. The verification half stops being tied to "I have the daemon running locally" and becomes something CI can invoke on a human-authored PR.

Developer impact: the evidence-based review gate becomes usable on code the agent never touched. That's a meaningfully different product.

Example: run it on PRs into a shared branch and let the tamper/repro/reviewer output land as a checklist beside your existing CI.

Windows fixes, which are a category and not a footnote

A task whose title or output contains Hebrew, Cyrillic or Japanese text no longer crashes at commit and strands the work. A .env saved with Windows line endings no longer reads as empty and silently hides your credential. The coder no longer loses all codebase context from a drive-letter path.

Why it matters: every one of these is an invisible failure. An empty credential file that reads as absent, or a coder silently running without repo context, produces a bad outcome with no error. Those are the same failure class the verification gates exist for, at the infrastructure layer.

Also in this release

The running version is shown in About from a single source.
nh task add --follows records that one task supersedes another.
Signing, stated plainly: macOS is signed, notarized and stapled. Windows is NOT code-signed — the artifacts carry -UNSIGNED, SmartScreen will warn, and you should check against SHA256SUMS-windows.txt. Linux ships a .deb and an AppImage with checksums.
The release publishes its own known issues, including a residual Windows non-UTF-8 codepage read path due in 0.2.5.

A release that publishes its own known issues is a small signal, but it's the same signal as the rest of the project.

Try it yourself

You need a Claude credential and the Claude Code CLI, whichever way you install — the backend shells out to that CLI for every task.

# prerequisites
npm install -g @anthropic-ai/claude-code
claude setup-token

# install (CLI + board)
uv tool install no-human        # or: pipx install no-human

# initialise, then prove the install is real
nh init && nh doctor

# run
nh start                                              # board + worker on 127.0.0.1:8420
nh task add https://github.com/org/repo/issues/42 --repo ~/git/repo
nh status                                             # needs-you / working / waiting / done
nh review <id>                                        # the reviewer's evidence checklist
nh diff <id>                                          # the diff it wants to ship
nh approve <id>                                       # your approval squash-lands the PR
nh reject <id> --reason "..."                         # send it back with feedback

From source, you also need Python 3.12+, uv, git, and Node with npm — the board is a separate npm run build, and a source checkout ships no web/dist.

What to expect: a task appears on the board, gets a plan, gets implemented, runs your tests, goes through the gates, and stops as an open PR in awaiting_approval. Run nh review <id> before nh diff <id> — reading the evidence checklist before the diff is the habit the whole design is trying to build.

Already inside an agent? There's an MCP server — a stdio bridge on the official Python MCP SDK — exposing exactly two tools, task_add and task_status. It talks to your own no_human at 127.0.0.1:8420 and nothing else.

nh mcp-serve

{
  "mcpServers": {
    "no_human": { "command": "nh", "args": ["mcp-serve"] }
  }
}

For Claude Code, the repo is its own plugin marketplace:

/plugin marketplace add no-human-ai/no_human
/plugin install no-human@no-human-ai

Docs worth reading before you rely on any of it: verification.md (the gates and their limits) and security.md.

When this approach makes sense

The project is explicit that ambitious tasks are not the target. It's aimed at well-scoped work: bugfixes, test gaps, small features, investigations. A vague ticket produces an escalation, and that's the intended behaviour rather than a workaround.

Good fits:

Bug fixes with a reproducible failure — the repro gate is doing real work here
Test-gap and coverage tickets
Repetitive maintenance across a repo
Investigations where an honest "here's what I found and here's what I can't determine" is a valid deliverable
Teams who want to experiment with agent workflows without giving up a human merge gate
CI-integrated workflows, especially now that the review gate runs as an Action

Where a human still has to be in it:

Architectural change, where the acceptance criteria aren't expressible as tests
Security-sensitive code — note that a review gate which executes anything repo-authored is a location for exposure, and the project says so in its own docs
Dependency changes, where the blast radius isn't in the diff
Anything where the ticket is the actual problem. No gate fixes an incomplete acceptance criterion.

What I'd watch carefully

A balanced read requires saying the uncomfortable parts, and to the project's credit most of these are documented by the project itself.

The reviewer is a model, and models miss things. The published confirmation run (2026-08-11, claude-opus-4-8, 19 seeded defects + 10 controls) measured recall 15/19 (79%) and specificity 7/10. That's a useful gate, not a proof. It also means false positives are routine — plan for triage.
Different models can share blind spots. Independence of session is structural; independence of reasoning is not. Two models trained on overlapping data can agree confidently and wrongly.
"Different model" is a default, not an invariant. You can configure the reviewer and implementer to the same model. Nothing stops you. That would quietly delete the property the whole design rests on.
The read-only reviewer isn't structurally read-only. The docs say this plainly: the edit tools are refused, but Bash isn't, so a shell redirection could still write a file. The refusal happens at the tool call, in a guard that reads a command line — a cost, not a proof, and the modelled set of spellings is not closed.
The benchmark is self-run and you can't reproduce it. The harness is reusable; the corpus pins to the author's local paths. Success rate moves several points between runs on identical specs, because the coder is non-deterministic.
No dollar figure is a billed number. Spend is capped in cost-weighted tokens; cache reads were 95.6% of tokens burned in the project's own lifetime measurement. Treat any cost estimate as an estimate.
Automated tests are a ceiling. Everything here verifies against the tests that exist. A gate can prove a test failed at base and passes now; it cannot prove the test was the right test.
Language coverage is uneven. Tamper guard reads Python, JS/TS and Java. Repro gate defaults to pytest.
There is no deploy step. The pipeline ends at an open PR, deliberately.

None of this makes the approach less interesting. It makes it legible — which is the same property the gates are chasing.

The bigger question

The industry's question right now is how much code can AI write? We have an answer: a lot, quickly, and getting faster.

I think the more useful question is:

How much engineering work can AI perform while producing evidence a human can independently check?

Because the second question is the one that determines whether any of this scales past a single motivated developer watching every diff. Volume of generated code is bounded by review capacity. And review capacity isn't bounded by reading speed — it's bounded by how much you have to reconstruct before you can judge.

An agent that hands you a diff makes you reconstruct everything. An agent that hands you a diff, a reproduction that fails at the merge base and passes on the head, a test-integrity delta, a fresh-context reviewer's checklist with verified file:line citations, and a per-rule merge verdict has done part of your job rather than just part of its own.

That's what no_human is actually building, and it's why I found it worth three PRs and three rounds of review that were harder than most code review I've had from humans.

The name is a joke, incidentally. The entire architecture is organised around a human saying yes.

If you're interested in trustworthy AI-assisted development

The project is MIT, runs locally, and is genuinely open to outside work — v0.2.4 shipped contributions from five people outside the core project, each with a CLA entry under contributors/ and their own name on their commits.

Places to start that aren't "write code":

Read docs/verification.md. Even if you never install it, the " and what it does not cover" half is the best short document I've read on the limits of automated verification.
Read an open PR thread. The review culture is the product.
Run it on a small bugfix in a repo you know well, and read nh review <id> before the diff.
Open an issue about the failure modes your codebase has that these gates wouldn't catch. That's the most useful thing you can contribute to a verification project.
If you build agent tooling yourself: steal the reproduction gate. It's twenty lines of concept and it's the highest-value idea here.

Project site and docs: https://getnohuman.com/

GitHub: https://github.com/no-human-ai/no_human

Release v0.2.4: https://github.com/no-human-ai/no_human/releases/tag/v0.2.4

If you try it, I'd genuinely like to know which gate fired first on your repo — and whether it was right.

Credits

no_human is built and maintained by Eyal Golan (@eyalgolan). Thanks to him for the invitation to write this up, and for three rounds of review that were more rigorous than the code they were reviewing. Any mistakes in my description of the project are mine; everything I got right about it, I got right because the documentation and the review comments said so plainly, including the parts that weren't flattering.

Top comments (3)

Sunil Kumar Uikey • Sep 17

Writing the code is increasingly becoming the easier part. The harder problem is proving that the change actually fixes the intended issue without introducing something else.

I think the verification step needs to be treated as a separate task: tests, edge cases, regression checks, and clear acceptance criteria. An AI-generated fix can look convincing while still failing in less obvious scenarios.

This is probably where AI-assisted development will mature — not just generating code faster, but making the validation process equally systematic.

Prince Panchani • Sep 17

Absolutely agree!

I think the next evolution of AI-assisted development is exactly this: from code generation to evidence-backed verification. Great point!

Piekwerk • Sep 17

The reproduction gate is the idea I'd steal too, with one scar to add: rerun the base-fail check before trusting it. I once had a fix where the offered test failed at the merge base about every third run. Flaky, not causal, and the single run our check sampled looked like perfect evidence. Running the old tree three times and requiring three failures cost seconds and caught it. The other habit I keep regardless of gates is reading the assertion diff first. A changed assertion is still the cheapest signal there is, and it scans in seconds next to a full reviewer pass. Gates raise the floor. They don't replace that reflex.