Every codebase has rules nobody wrote down. One package always checks rows.Err() after iterating. Another logs through slog and never log.Printf. Every method on Store names its receiver s. Tests in this directory use testify; tests over there use the standard library.
AI coding agents break these rules constantly, not because they're careless but because they cannot see them. The rules live in the statistics of the code, not in any file. The standard fix is to write them down in a CLAUDE.md or AGENTS.md. Someone has to write it, keep it current, and hope it covers what matters. A typical hand-written file has 20 or 30 rules for an entire repository. The repository has thousands.
I spent the summer trying to close that gap automatically. The first two attempts failed, and the way they failed is the useful part.
Attempt 1: give the agent a tool
The obvious design in 2026 is an MCP server. Index the repo, mine its conventions, and expose one tool, get_style_context, that takes a file path and returns the rules, tendencies and example functions for that file's directory. The agent calls it before editing, reads the rules, writes conforming code.
I ran a controlled experiment: 15 runs, same tasks, the tool available every time. The agent called it zero times.
I assumed the tool description was the problem and rewrote it. More imperative. More specific about when to use it. Still zero. The agents did what they always do: read the file, read a neighbour or two, write code. They had no signal that they were missing anything, so they never went looking.
That's the general lesson, and it's bigger than my tool. An agent will not voluntarily look up knowledge it doesn't know it lacks. A knowledge-lookup tool that depends on the agent's initiative is a tool that doesn't get used.
Attempt 2: push the rules at it
If the agent won't pull, push. Claude Code has a PreToolUse hook that runs before an Edit or Write. I used it to inject the relevant conventions right before each edit landed.
Six blind-scored runs on Kubernetes with Claude Opus. The agent wrote the same code with or without the injected rules.
In hindsight the reason is obvious. By the time PreToolUse fires, the agent has already authored the edit. The content is in the tool call. Rules that arrive at that moment are advice about a decision that's already been made.
Attempt 3: stop telling, start checking
The thing that works is boring. Don't try to inform the agent at all. Let it write the code, then check the code, and if it breaks a rule hand the violation back as an error.
OpenTuple does this with a PostToolUse hook on Edit|Write. After every edit it re-parses the file with tree-sitter, works out which directory each function belongs to, and compares the function with the conventions mined from that directory. If something breaks a rule, the hook exits with status 2, which is the code Claude Code uses to show a hook's stderr to the agent. The agent sees this:
internal/cli/summary.go:9: PrintIndexSummary calls fmt.Fprintf but never Render
Rule: In internal/cli, functions that call fmt.Fprintf also call Render. [helper_pairing]
Evidence: 12 of 12 functions in internal/cli that call fmt.Fprintf also call Render (100%); 4% of the functions that do not call fmt.Fprintf call Render.
Example: reportWrittenFiles (internal/cli/generate.go:218) follows this rule.
That's from a real run with Claude Haiku. It was asked to add a summary printer to a CLI package and its first version printed with a bare fmt.Fprintf. Every other printer in the package styles its output through the theme. The hook said so. The agent's next edit wrapped the text in StyleDim.Render(...), the check passed, the session ended. Four turns, nine seconds. Nobody told it about the theme. The repository did.
Why does this work when the first two didn't? Because an agent treats a hook error the way it treats a failing test. It's not advice it can weigh and discard. It's a blocker it has to clear. The timing that made attempt 2 fail (the hook fires after the content exists) is exactly the timing a check needs.
The hard part was silence
Detection turned out to be the easy half. The hard half was not flagging things that aren't wrong.
A convention checker lives or dies on false positives. The first time it complains about something that was fine, the developer learns the check is noise, and the second time they turn it off. So OpenTuple's rules are deliberately strict:
- A rule exists only where at least 98% of a directory's functions follow it, with at least 10 of them.
- A helper pairing (functions that call
Lockalso callUnlock) has to beat the paired helper's base rate by 30 points. Otherwise, in a package where everything callsErrorf, everything that callsClose"pairs" withErrorftoo. - An absence rule ("nothing here calls
panic") needs enough functions that you'd expect some to call it at the repo-wide rate. Ten functions that never neededCloseis a small sample, not a policy. - Test files are their own scope, so a convention of your tests is never imposed on your library code.
The measure of this is how quiet it is on code that already follows its own conventions. Running the check across whole repositories:
| Repository | Files | Flags |
|---|---|---|
| fzf | 79 | 0 |
| GitHub CLI | 860 | 3 |
| Kubernetes v1.29 | 7,845 | 49 |
| 10 JS/TS repos | 3,191 | 0 |
The 49 in Kubernetes are real minority deviations: 31 test functions using a different assertion library from the rest of their package, 11 doc comments, 3 receiver names, 2 pairings, 1 logging call outside klog, 1 error-handling site.
What it looks like to use
go install github.com/CtrlAltDelete-Ops/opentuple/cmd/opentuple@latest
cd your-repo
opentuple init .
init indexes the repo (Kubernetes, 9,711 files, takes 27 seconds on an M3), writes a short CLAUDE.md per directory and one AGENTS.md for tools that read a single file, and installs the hook in .claude/settings.local.json. Then you start a Claude Code session and forget about it. Not on Claude Code? --git-hook runs the same check on staged files as a pre-commit hook.
One Go binary. Runs locally. No network requests. Apache-2.0. Go, Python, JavaScript, TypeScript, Rust and C.
What I don't know yet
I'll be upfront about the gaps. .tsx files go through the TypeScript grammar, which has no JSX, so React components are only partly seen. There are no prebuilt binaries yet, so you need Go installed. And my behavioural numbers come from small runs. I know the mechanism works; I don't yet have a large, independently graded measurement of how often it fires on real agent-written code, and I'd like help designing one.
The repo is at https://github.com/CtrlAltDelete-Ops/opentuple. If you run it on your codebase, the thing I most want to hear is what it flagged and whether it was right.
Top comments (0)