I stopped writing rules for coding agents that CI could not enforce

I used to treat repository instructions as the finished product.

Write a careful AGENTS.md. Explain the architecture. Ban any. Require named
exports. Tell the coding agent to keep files small and update the ADR when a
contract changes.

The document could be excellent and the next pull request could still violate
it.

That is not always the agent's fault. A rule that exists only as prose asks
every contributor, human or automated, to remember it at exactly the right
moment. Review becomes the first place where the repository discovers that the
rule was ignored.

I started treating the instruction as the explanation and a gate as the
enforcement.

Start with one failure, not a governance rollout

The tempting response is to automate everything. That creates a different
problem: a wall of noisy checks that nobody understands and everyone learns to
bypass.

I now start with a failure that has already happened.

Suppose an agent uses any to get a TypeScript change through quickly. The
repository rule says to use unknown, narrow the value, and validate external
input at runtime. The reviewer catches the shortcut, asks for a revision, and
the same mistake returns two pull requests later.

That is a good gate candidate because the rule is:

• repeated;
• mechanically detectable;
• cheap to check;
• actionable when it fails.

“Choose the right abstraction” is not a good gate candidate. It still needs
engineering judgment.

Reproduce the smallest useful version

I published the executable gates from
Agents Playbook as a
zero-dependency CLI. You can test one gate without adopting the rest of the
playbook.

Create a small fixture:

mkdir -p /tmp/playbook-gate-demo/src
printf 'export const answer: number = 42\n' \
  > /tmp/playbook-gate-demo/src/example.ts

Run only the no-any gate:

npx --yes @agentskit/playbook@0.1.0 \
  run no-any \
  --cwd /tmp/playbook-gate-demo

The valid file passes:

no-any: OK. 0 escape-hatched.

Now replace it with the shortcut:

printf 'export const unsafe: any = 42\n' \
  > /tmp/playbook-gate-demo/src/example.ts

npx --yes @agentskit/playbook@0.1.0 \
  run no-any \
  --cwd /tmp/playbook-gate-demo

The command exits non-zero and points to the violation:

src/example.ts:1:22 — `any` in type position.
Use `unknown` + a runtime schema parse, or a specific type.

The useful part is not that a script can search for any. Mature TypeScript
repositories should normally use the AST-based
@typescript-eslint/no-explicit-any rule. The useful part is the shape of the
feedback:

1. the repository states the rule in language a contributor can understand;
2. a fast check catches the mechanical part;
3. the error explains the preferred replacement;
4. the same command runs locally and in CI.

The policy and the gate have different jobs, but they point in the same
direction.

Escape hatches need a budget

An absolute rule often becomes dishonest.

There are boundaries where a legacy dependency, generated type, or migration
really does require a temporary exception. If the only available responses are
“rewrite the dependency” or “disable the check,” people disable the check.

The reference gate accepts an explicit same-line exception:

export const legacyValue: any = input // allow-any: upstream SDK has no types

That comment is not a magic phrase that makes the type safe. It makes the
decision visible. The gate counts escape hatches, and a repository can keep a
baseline so the number does not quietly grow.

This is the difference between an exception and an invisible bypass:

• the reason sits next to the code;
• reviewers can challenge it;
• CI can prevent exception growth;
• the team can search for the debt later.

A gate that cannot represent a legitimate exception will eventually be worked
around outside the gate.

Put the check before the expensive feedback loop

I want the shortest useful loop to be available before a pull request.

For a repository using the Playbook CLI, the package scripts can stay small:

{
  "scripts": {
    "check:agent-rules": "agents-playbook run no-any named-exports",
    "precommit": "pnpm check:agent-rules"
  },
  "devDependencies": {
    "@agentskit/playbook": "0.1.0"
  }
}

CI should run the same command. A pre-commit hook is a convenience, not the
trust boundary: hooks can be skipped, while the required CI check protects the
shared branch.

The fast local path matters for coding agents in particular. If an agent can
run a focused gate immediately after editing, it can repair a violation while
the relevant context is still active. Waiting for a large repository pipeline
turns a five-second correction into another review cycle.

Do not confuse a green gate with a good change

The no-any gate can prove that it did not find the pattern it recognizes. It
cannot prove that the replacement type is correct.

The named-export gate cannot prove that an API is well designed. The file-size
gate cannot prove that splitting a file improved cohesion. A secret scanner
cannot prove that authorization is correct.

This boundary is important when agents are involved. Machine-checkable rules
are attractive because they produce a clean pass or fail, but engineering
quality is larger than the set of things that are easy to count.

I use three categories:

Rule	Best enforcement
Deterministic syntax or repository invariant	Automated gate
Contextual design choice	Review checklist and examples
High-impact ambiguous decision	Explicit human approval

If a design rule cannot be checked without guessing intent, I leave it in the
review layer. Automating a weak proxy can be worse than admitting that a human
decision remains.

The adoption sequence that held up

The Playbook currently includes 13 zero-dependency reference gates covering
concerns such as secrets, file size, named exports, ADR/RFC requirements,
internationalization, design tokens, native HTML, and PR intent.

I would not enable all 13 on day one.

The sequence I use is:

1. collect one recurring failure from a real review;
2. write the repository rule and the reason behind it;
3. add the smallest gate that catches the mechanical violation;
4. make the failure message tell the contributor what to do next;
5. run it locally and in required CI;
6. measure false positives before adding another gate.

That keeps governance attached to evidence. Every check should be able to
answer: “Which failure are you here to prevent?”

I created and maintain Agents Playbook, so this is not a neutral tool review.
The CLI and all 13 reference gates are open source, and the narrow experiment
above is intentionally easy to reproduce without an API key or model call.

Preparation disclosure: I used AI tools to help organize and critique this
draft. I ran the commands against the published package, inspected the gate
source and tests, checked the claims against the public repository, and stand
behind the final text.

If you already enforce a coding-agent rule in CI, I am interested in the rule
that produced the most useful failure message—not the largest policy file.