Guardrails for AI-Assisted Development: Skills, Gates, Hooks and Mutation Tests

When a model writes the code, writing it stops being the expensive part. Proving you didn't break something nobody was looking at becomes the expensive part.

That sentence took me a few months of building to arrive at, and once I had it, most of my tooling decisions stopped being arguments.

This is a walkthrough of the verification layer in a real project of mine: a wellbeing analytics platform with a Next.js core and a Python AI service. Four agent skills, sixteen repository gates, a hook that runs them at edit time, and a narrow mutation testing pilot. Everything here is running code, and I'll be specific about what each piece does not catch, because that turns out to be the more useful half.

Why conventions in a README don't survive

Let me start with the failure that convinced me.

My repo has an AGENTS.md saying that .agents/skills/ is the single source of agent instructions. A month earlier, GEMINI.md had said in plain prose: don't create client-local copies of these skills.

A 3,442-byte partial copy of one skill showed up in .gemini/skills/ anyway. Two weeks later there was still an empty leftover directory next to it.

Here's the part that matters: .claude/ and .gemini/ are in .gitignore. The copy was invisible in every diff and every code review. Not missed by a reviewer. Invisible to the mechanism of reviewing.

Two copies both claiming to be canonical, drifting apart with nothing comparing them. Prose has no enforcement mechanism. That's not a discipline problem, it's a category problem.

Layer 1: a skill tree with exactly one root

Skills are markdown files that tell an agent how to work in this repository; mine cover product invariants, session/branch state, verification, and the gates themselves. The design has four rules.

One canonical location. .agents/skills/ and nowhere else. There is no list of skills in any config file. The canonical set is the directories on disk:

const skillNames = fs
  .readdirSync(SKILLS_ROOT, { withFileTypes: true })
  .filter((entry) => entry.isDirectory())
  .map((entry) => entry.name);

A folder appears, a skill exists. This is why a copy elsewhere is dangerous rather than merely redundant: if the filesystem is the source of truth, any folder with a familiar name becomes a second source of truth.

Routing in the entrypoint. AGENTS.md carries a plain markdown list mapping tasks to skills:

- Verifying, testing, proving a fix, checking readiness, reviewing evidence, or
  about to claim a substantive change is complete →
  `.agents/skills/shalomut-verification/SKILL.md`.

Why bother if some clients discover skills automatically? Because a client with discovery and a human reading the file directly must arrive at the same rules. Discovery changes how a file is found, never what it says.

A reading map inside each skill. Every skill has a How to read this skill section splitting its own sections into "always in force" and "open when this condition holds." That map, not the entrypoint, decides how much of a skill a task needs. The rule next to it: loading a section is cheap, skipping a rule is not, so when a condition is ambiguous, open the section.

A check that all of the above holds. npm run lint:skills refuses: a skill copy outside the canonical root (including an empty directory named after one), a broken or orphaned link in references/, a ## section the reading map never classifies, and any client entrypoint that routes nowhere.

Layer 2: gates, or how a convention becomes unignorable

A gate is a script that reads the repository and exits non-zero when it finds a violation. Nothing executes. No arguments get fuzzed, no mutants get generated. It reads — not just paths, but anything knowable without running the program: where a repository gets constructed (lint:composition), what a response body contains (lint:error-bodies), whether a document still agrees with the configuration it quotes (lint:doc-numbers).

What actually deserves a gate

This is the decision rule I'd most want to hand to someone starting out, and it took me a while to get right:

Kind of rule	Where it belongs
Lives inside a module, expressed through its API	An ordinary test. No gate.
About the shape of the repository: what may be imported, where a literal may stand, which interpreter runs, where skills live	A gate
Editable source with a derived copy	A --check mode on the generator, not a second equality test
A machine cannot judge it: is this architecture right, is this audit record genuinely closed	Prose. And say so in the gate's doc comment, so a green gate never reads as proof of what it never checked

The tell that you need a gate: the violation is silent. Tests green, build passing, reviewer sees nothing, and the rule is already broken. If a violation fails the suite anyway, a test is enough and a gate is overhead.

One gate in full

The audit of my repo counted 21 route handlers interpolating a raw error.message into what they sent back. On /api/auth/login that went to anyone. Every one was written by somebody being helpful.

The naive check is a regex for error.message. I wrote that first. It let (error as Error).message straight through, and the audit itself had missed error?.message for the same reason.

So the rule became two rules:

1. a catch in a route handler binds the name error;
2. the argument of NextResponse.json(...) never mentions it.

Rule 1 exists to make rule 2 complete. Refusing the whole identifier makes the spelling irrelevant, and an identifier rule is only as good as the identifier, so rule 1 stops a handler slipping past with catch (e).

The implementation is not a regex either. It's three passes:

// 1. Strip strings and comments, but KEEP ${...} inside templates.
//    Without stripping, { error: 'Internal error' } fails on its own wording.
//    Stripping templates whole would let `failed: ${error}` through,
//    and that is the leak itself.
export function stripTextAndComments(source) { /* hand-written scanner */ }

// 2. Find the argument region of a literal call by counting parens.
export function argumentRegions(source, call = 'NextResponse.json(') { /* ... */ }

// 3. Only now, a pattern. A property access is fine (`produced.error` is our
//    own field). A spread is not: { ...error } puts the whole thing in the body.
new RegExp(`(?<![\\w$])(?<!(?<!\\.)\\.)${BINDING}\\b`).test(readable)

Two places in the codebase used the name error for the product's own refusal wording. They got renamed rather than exempted. An exemption list is a place the next leak hides.

And the blind spot, stated in the doc comment rather than discovered later: a body assembled into a variable and passed by name goes unnoticed. That gap is accepted, because the alternative is parsing TypeScript in a fitness check and every occurrence the audit found was a literal at the call site. With the follow-up that matters: if that stops being true, this needs a parser, not a wider regular expression.

The shape every gate shares

// scripts/check-<subject>.mjs
/**
 * <The rule. Why it exists — name the actual incident. What this cannot see.>
 */
export function findViolations(source, file) { /* pure, returns string[] */ }

function main() { /* reads files, prints, process.exit(1) */ }
if (process.argv[1] === fileURLToPath(import.meta.url)) main();

Pure exported functions so the test can import them. main() guarded so importing doesn't run it. And on success, a line saying how much was checked:

Error-body fitness check passed: 47 route handlers, no caught error in a response body.

A silent success is indistinguishable from a check that read nothing.

Wiring, and why every gate has its own test

"lint:error-bodies": "node --test scripts/check-error-bodies.test.mjs && node scripts/check-error-bodies.mjs",
"verify:core": "npm run lint:literals && ... && npm run typecheck && npm test && npm run lint && npm run build"

The gate's test runs before the gate. That order is the whole point: prove the check can fail, then trust it. A check whose pattern never matches anything is worse than no check, because it manufactures confidence.

Gates that guard the gates

The cheapest way to fix a red gate is to delete it from the chain. So:

lint:gate-inventory refuses:
  - a gate that isn't a step of verify:core
  - a gate missing from the inventory table
  - an inventory row with no gate behind it
  - a lint:* command that doesn't run its own test

Without this, the whole safety system rests on good faith, which means it doesn't rest on anything.

Same principle inside the skills: the guardrails skill says outright that weakening a check, adding a file to an exemption list or narrowing its scope is a change to the rule, not a fix to the build — do it deliberately, update the doc comment and the tests on both sides. And: never bring a gate's test in line with current behaviour to make things green. The gate's test is the record of the rule.

Layer 3: the hook, so the agent hears it now

CI is the record. But CI tells you at 11:00 about something written at 10:00, by which point it's buried under later edits made on top of it.

.claude/settings.json is the one file under .claude/ that Git tracks — .gitignore un-ignores exactly it:

{
  "hooks": {
    "PostToolUse": [{
      "matcher": "Edit|Write|MultiEdit|NotebookEdit",
      "hooks": [{ "type": "command", "command": "node scripts/gate-hook.mjs", "timeout": 90 }]
    }]
  }
}

gate-hook.mjs holds no rules. It takes the path of the file just written, maps it to the gates that could care, and runs those:

const RULES = [
  { gate: 'lint:error-bodies',
    when: (file) => /^src\/app\/api\/.*\/route\.ts$/.test(file) },
  // ...
];

On refusal it exits with code 2, which feeds stderr back to the agent, and the message points to the relevant skill section and explicitly says not to weaken the check or rewrite its test to make it pass. That last line is defence against the first thing an agent tries.

Three limits, all stated in the file itself:

• 15 of 16 gates are mapped. lint:literals is deliberately out: its Python half needs an interpreter from .venv, and a missing local environment must not read as a violation.
• The mapping is narrow on purpose. A gate that fires on every edit is a gate people learn to wait out.
• It runs in one client only. A rule that lives in a hook doesn't exist for Copilot, for Gemini, or for a person in a terminal.

One implementation detail worth stealing: the hook runs only the check half of each command, not the node --test half. The first version buried a one-line violation under sixty lines of passing TAP.

Layer 4: tests, and proving the tests are worth anything

Gates check shape. Tests check behaviour. Neither checks whether the tests are any good. That's mutation testing: deliberately corrupt the code, see whether tests notice. Mine runs on two files — the AI contract validator and the scoring bands — and the full run isn't a CI gate: the score moves when a function migrates between files or a test file enters the list, which has nothing to do with test strength.

The checks around it keep that number meaningful. lint:mutation-config re-derives tap.testFiles from the repo, because a missing entry doesn't lower the score honestly — it reports as survived a mutant a real test would have killed. lint:contract-refusals demands a negative-test suite for every contract version; it proves a suite exists, not that it is complete. And when a product rule leaves a mutated file, mutate follows it in the same change, or the rule silently drops out of measurement.

Layer 5: how much checking is enough

This is the part I'd defend hardest, and the part most teams skip.

My verification skill carries a selection matrix: rows are areas a diff touched, cells are the mandatory minimum. The principle: choose the smallest set of checks that proves the changed behaviour, then widen in proportion to risk.

A few rows:

Changed	Mandatory minimum
Markdown / skills only	Frontmatter and links, git diff --check, lint:skills
A repository gate	Its paired test and the gate, lint:gate-inventory. Weakening the check = changing the rule
src/app/api, services	Nearest tests, then npm test and npm run build
AI contract	lint:contract-refusals, contract/registry/client tests, Python tests, local boundary E2E
Python dependencies	lint:python-deps; production runs 3.11 and dev machines usually don't have it, so the one real proof is docker build plus the suite inside that image
Auth or secrets	Unauthorized / missing-secret / tenant-isolation tests plus a security-focused diff review

Running everything for a docs edit is the same mistake as running one small test for an auth change. Both look like diligence. Neither is.

One check belongs to no row: typecheck is mandatory for any .ts change. build types only the application graph, lint doesn't check types at all, and npm test runs through tsx, which strips types without checking them. A green test run tells you nothing about types.

Say what counts as evidence

With an agent this stops being pedantry. Models report affirmatively by default, so the rules are written down:

• Label the context — local, test, or deployed — and never mix them silently.
• npm run dev starts a runtime. It is not evidence.
• A mock MCP server is not proof of real transport, and a wrapper that delegates to the same command is not a second piece of evidence.
• Don't promise repo-wide mutation coverage. Separate killed / survived / no-coverage / runtime-error.
• verify:core is an && chain: it stops at the first failure. The steps after it did not run — don't report them as passed.

Every rule carries its incident

The habit I'd transplant into any codebase regardless of AI: each check's doc comment names what broke, with a date.

• Fonts. Until 2026-08-12 the build downloaded five .woff2 files from a Google host; when a runner got a stale stylesheet, all five 404'd and the build failed with a message mentioning neither fonts nor the network. The same commit built cleanly in the neighbouring job, so the gate had become a coin toss instead of a red light.
• Tenant chokepoint. A context loader recorded the visit from the request rather than the answer, so an administrator reading the only tenant left no audit row at all, and no test failed.

The corollary is a cleanup rule: a rule with no incident behind it is a candidate for deletion. Either it isn't needed, or nobody remembers what it defends — and the first person it inconveniences will remove it.

What this doesn't do

• Gates read text. A determined rename or a dynamic call walks around them. They defend against accidental violation, not adversarial.
• Presence checks don't check agreement. A client adapter passes if it contains the string .agents/skills, even if the surrounding sentence says the opposite. Stated in the code; it stays a review question.
• None of this judges whether an architectural decision is right.
• It costs. Sixteen gates is sixteen scripts plus sixteen tests plus an inventory plus a gate guarding all of it. On a two-week prototype that's absurd. In a regulated domain where the question is "prove this data couldn't leak," it's the cheapest answer I know.

If you're starting tomorrow

1. Take the last silent bug you shipped. Write the smallest script that would have refused it. One rule.
2. Give it a test that proves it can fail, and wire the test to run first.
3. Put it in one verify chain with everything else, and put that chain in CI.
4. Write the incident in the doc comment, with the date, and write down what the check can't see.
5. Only then hook it into your agent's edit loop, so the refusal arrives while the context is still warm.

Steps 1 through 4 work with no AI involved at all. Step 5 is just the feedback arriving earlier. That ordering is deliberate: the reason any of this holds is that the rules live where every client and every human reads them, and the agent integration is the last mile rather than the foundation.