AI writes code fast now. Reviewing it has not got faster.
On the production Django backend we used as a test bed, about half of the recent commits were fixes. The same rule lived in several copies: sign-in, sign-out, password change and JSON body parsing each had more than one version, and they had drifted apart. One access bug came back a week after it was fixed.
So a colleague and I built healthkit, an open-source toolkit that checks Python code for the problems a tool can find reliably, so people (and AI reviewers) can spend their time on the ones it cannot.
The most useful thing we did was not a feature. It was measuring.
The number that shaped everything: 9 out of 45
A careful human review of one module had produced 45 findings. We replayed it: take the code from before each fix, run our rules, and see which findings a rule points at.
The first score was 29 of 45. It looked great. Then we read every hit by hand. Most were coincidences: an unused import that happened to sit inside a behaviour fix, findings inside routes the fix deleted anyway. Counting only the hits where the rule pointed at the actual problem: 9 of 45, about 20%.
That gap is the first lesson. Distrust the metric that looks good. Every automated score gets a by-hand read of its hits before we quote it.
The second lesson came from breaking the 45 down:
| Kind of problem | Example | Caught by rules |
|---|---|---|
| Pattern: a shape in one place | an exception's text sent back to the client | 8 of 12 |
| Knowledge: needs the project's model | reading rows that were soft-deleted | 0 of 13 |
| Behaviour: does it do the right thing | a reset link that works as a login; a race | 1 of 15 |
Rules are good at patterns. They are close to useless at behaviour. Pretending otherwise is how a green check turns into a false sense of safety.
What a tool can own, and what it never can
That table became the design. A rule is a past mistake written down so a machine can spot it again. It catches known kinds of mistakes, never new ones.
- Patterns and structure (who may import whom): healthkit owns these, using ruff, Opengrep and import-linter with rule sets we measured.
-
Project knowledge: partly, through small guards each project writes in its own
.healthkit/folder. - Behaviour: never by rules. That is the job of tests, QA and people.
So the README says it plainly: healthkit can never judge whether the logic is right, whether it is secure, or whether the design is right. A clean run means "nothing it knows about", never "safe".
Five design decisions
1. AI is not in the gate. The pass/fail decision uses deterministic rules and tools only. AI review is advisory and never blocks. When AI spots a recurring problem, it becomes a rule, and the rule enforces it.
2. Check the lines you changed. healthkit review --base main checks only the lines your branch changed, so an old codebase does not bury you in history. scan and baseline exist for when you want the whole module.
3. The agent writing code cannot quietly change the referee. Most runs are done by AI coding agents before they hand over work. An agent that can edit the rules, add exceptions or exclude folders can make any check pass. So any change to the .healthkit/ settings, and any # noqa that switches a check off, is flagged as needs human approval, even when everything else is clean. We found the gap by hand: the default excluded everything under migrations/, so real code placed there would never be checked. The default now matches only Django's generated migration names.
4. Weak rules warn, they don't block. One rule found real bugs but was right only about half the time on code it was never tuned on (41% and 55% on two projects). It ships as worth a look:: printed and counted, but it never fails the run. A rule moves to blocking only after reaching 70% on unseen code from at least two projects.
5. "Could not check" is never a pass. Exit codes: 0 nothing found, 1 problems, 2 could not check, 3 only a settings change that needs approval. This came from a tool that reported zero unused functions on code with known dead code. Its verification step had timed out and it quietly withheld everything. "0 findings" can mean "did not run". Every adapter now surfaces the tool's own errors next to its findings.
Catching bugs nobody wrote a rule for
Behaviour bugs need a different approach. healthkit api sends good and bad input to every route of an API (via Schemathesis) and treats any server error as a finding. On the test project it found six real crashes, none of which had a rule.
It also answers a question rules can't: can this user see that data?
- Multiple users: you give it one token per role, for example an admin, a plain user, and a user in another organisation. Every route is tested as each of them, and it checks that one user cannot read another's objects.
- Access policy: you write down who must be refused where, and any successful answer is a finding:
[[api.deny]]
user = "HK_TEACHER"
routes = ["GET /people/students", "GET /finances/**"]
reason = "a teacher sees only her own class"
-
The size of a leak: a finding says how much came back (
got 200 (469 records)orgot 200 (empty)), so a real leak stands out from an empty list. It shows only the count, never the data. -
Read-only by default: it sends GET requests only, unless you add
--allow-writes. We learned this the hard way: a sweep that wrote data deactivated the user that one crashing input needed, so the next sweep found 4 crashes instead of 6. Neither bug had been fixed.
Building it with AI agents: the lessons
healthkit was built with AI assistance under a written working agreement, one rule for each thing research says AI-written code tends to miss. Three lessons stood out:
- An agent's "refuted" is not evidence. A verifier agent marked a real bug as not a bug by misreading the code it quoted. It had been asked to run a test and did not. A two-minute script proved the bug. Now any claim, from any agent, counts only with output that was actually run.
- Verifiers drift to what is easy to confirm. Asked to try new ways around the gate, a verifier agent re-tested the gaps we had already closed and skipped the new ideas. The first new idea we tried by hand was a complete bypass. Now every verification brief lists the cases that must be tried, and the report is checked case by case.
- Measure before building, and you may build less. We planned a similarity engine to find copied code. Measuring first, it found 1 of 4 real cases. Two cheap checks of about 110 lines did better, and found a rate-limit bug and a security helper copied with the opposite behaviour, neither of which any review had named.
Try it
uv tool install git+https://github.com/paudelbaibhav/healthkit
healthkit review --base main
It works on any Python project, with extra checks for Django and web APIs. Output comes as text, JSON for agents, Markdown for pull-request comments, or GitHub annotations.
Code, decisions and every lesson we wrote down: github.com/paudelbaibhav/healthkit. It's MIT licensed, with about 220 tests.
What does your team rely on to review AI-written code: rules, AI reviewers, or people? I'd like to know what has actually worked for you.
Top comments (1)
The 29-to-9 correction is the most honest number in the piece — counting a hit only when the rule points at the actual problem is a discipline most tooling posts skip entirely. The pattern/knowledge/behaviour split explains why: rules own shapes, never behaviour, and a green check that pretends otherwise is false safety. The 'agent can't quietly change the referee' rule is the one I'd adopt first — has flagging .healthkit/ changes for human approval caught a real self-exemption attempt yet, or is it still insurance?