AI Development Rules Needed Regression Tests Too

I had already reached an uncomfortable conclusion:

Giving an AI coding agent better instructions in chat was not enough.

So I started moving those instructions into the repository itself.

In AIDDSkeleton, most of that governance lives in AGENTS.md files.

They define things like source-of-truth ownership, review behavior, evidence, scope, and lifecycle. Instead of reminding the AI about those rules in every prompt, I wanted an agent to read the repository, understand the rules, and make the right decisions on its own.

So the experiment was not:

Can I write a better prompt?

It was:

Can I change AGENTS.md and cause a fresh AI agent to behave differently without telling it what behavior I expect?

The repository became more consistent.

The agents seemed to behave better.

And then I ran into another problem.

How did I know the AGENTS.md changes were actually responsible for the improvement?

An AI agent can produce a better result for many reasons:

• the task prompt was slightly different,
• previous conversation context leaked the expected answer,
• the agent saw later commits,
• I accidentally hinted at the problem,
• or the model simply took a different reasoning path that day.

At some point, “this seems better” stopped being good enough.

So I started regression-testing the rules themselves.

---

The repository was becoming part of the prompt

The project is AIDDSkeleton.

It is a repository skeleton for AI-driven development.

Most of its interesting behavior does not come from scripts or a framework.

It comes from repository-local governance.

AGENTS.md files tell an AI agent how to interpret the information around it: what counts as authority, which documents are sources of truth, what evidence is sufficient, how findings should affect current scope, and when work should or should not continue.

That distinction matters.

I was deliberately trying to reduce the amount of intelligence carried by the immediate task prompt.

A task could remain relatively ordinary:

Review this change.

or:

Fix this issue.

The more interesting behavior was supposed to come from the repository instructions the agent discovered and followed by itself.

That creates a strange testing problem.

If I change a TypeScript function, I can run a test.

If I change AGENTS.md and expect an AI agent to make a different autonomous decision, what exactly do I run?

---

Commit 08ded669 — feedback stopped meaning “do this now”

08ded669 — docs: govern feedback triage and review recall

One early change separated two things I had been mixing together:

How serious is this finding?

and

What should we do with it now?

A review comment could be valid without automatically becoming current work.

The governance started distinguishing dispositions such as:

Accept now
Reject
Defer
Observe

from classifications such as:

Blocker
In-scope deficiency
Follow-up

This made it harder for every interesting suggestion to silently expand the task.

More importantly, this was not supposed to require me to remind the agent:

Be careful not to expand scope.

The rule lived in the repository.

The agent was supposed to encounter a finding, interpret it under the current AGENTS.md, and decide for itself whether that finding belonged in the current work.

That exposed a deeper problem.

I could read the new rule and think:

Yes. This is clearer.

But that still did not tell me whether a fresh agent would actually make a different decision because of it.

So I started applying candidate governance to another repository that already used AIDDSkeleton-style rules and observing what the agent did.

That became the basic idea:

A governance change is not validated only by reading AGENTS.md.

It also needs behavioral evidence from an agent actually operating under that AGENTS.md.

---

My first regression tests had a bug

At first, the experiments looked promising.

I would give the agent a review finding and ask things such as:

• Is there a common cause?
• Are there sibling surfaces with the same problem?
• Should this be reconsidered at a higher level?

The agent generalized.

It found related problems.

It produced broader corrections.

Great.

Except for one thing.

I had told it what kind of reasoning I wanted.

The candidate AGENTS.md might have been causing the behavior.

But the task prompt was carrying the same idea.

If I wanted to test whether repository governance caused an agent to look beyond the immediate symptom, I could not write:

Look beyond the immediate symptom.

I had effectively written the expected answer into the test input.

That meant the experiment could answer:

Can the model follow this hint?

But not:

Did the AGENTS.md change make the model do this on its own?

The regression test itself had a bug.

---

Commit 0611d61b — a finding became more than a patch target

0611d61b — docs: shift review learning into adversarial self-review

The review model then changed more substantially.

A valid finding was no longer just something to patch.

It could also reveal something about the reasoning that had failed to detect the problem earlier.

The idea was roughly:

finding
   |
   v
what did we miss?
   |
   v
does that missed perspective matter elsewhere?

Again, the important part was that I did not want to put that sequence into every review prompt.

I wanted a fresh agent to receive an ordinary finding, read the repository governance, and independently decide whether the finding implied a wider review surface.

That sounded useful.

It also made the testing problem harder.

If I explicitly asked the agent to inspect sibling paths or look for a shared cause, I would no longer know whether AGENTS.md itself was responsible.

So the prompt had to become more neutral as the repository governance became more capable.

---

Freezing the past

The next version of the experiment started from a real historical commit.

Not the latest repository.

Not a manually reconstructed example.

An actual past state.

I treated that commit as a fixture.

historical consumer commit
          |
          +-- Control
          |     old AGENTS.md
          |
          +-- Treatment
                candidate AGENTS.md

Both worlds started from the same repository state.

The task prompt was kept materially the same.

The intended experimental variable was the governance.

The important part was also what the agent was not allowed to see.

Everything after the fixture was treated as nonexistent:

• later commits,
• later Pull Requests,
• later review comments,
• known fixes,
• previous regression results.

In other words, I tried to create two artificial timelines from Git history.

The agent could see the past.

It could not see the future.

If a later review had already identified a bug, allowing the agent to discover that comment would destroy the experiment.

It would no longer be testing whether the candidate AGENTS.md helped the agent discover the issue.

It would be testing whether the agent could read the answer.

---

Control and Treatment did not behave the way I wanted

This was probably the point where I started trusting the method.

The candidate governance did not simply win every round.

In one MediaStorage scenario, the Control branch explored a collision problem more aggressively.

The original issue was effectively:

destination already exists

The Control agent spontaneously pushed further toward:

what if two writers race for the same destination?

The Treatment agent improved the requested surface, but in that run it did not expand as far into the race condition.

That was inconvenient.

It was also excellent evidence.

If every experiment had ended with:

new AGENTS.md good

I would have trusted the experiments less.

Instead, I had an actual behavioral concern.

The candidate governance seemed stronger in some kinds of generalization, but possibly weaker in immediate exploration breadth.

So I did not declare victory.

I changed the repository rules again.

---

This was regression evidence, not a benchmark

There is an important limitation here.

I was not running each fixture dozens or hundreds of times.

I was not estimating a statistically meaningful success rate.

And across different AI providers, I was not controlling sampling parameters such as temperature closely enough to claim that the runs formed a clean quantitative benchmark.

So results like:

Treatment expanded the review surface.

or:

Control found the race condition first.

should not be read as:

Treatment has a 73% probability of doing X.

I did not have that data.

The regressions were primarily adversarial behavioral checks.

I was asking questions such as:

• Can the intended behavior appear without being prompted?
• Does the candidate introduce a material behavioral regression?
• Does it cause scope or authority violations?
• Does a concern still reproduce after the governance is refined?
• Does the final candidate retain the behavior seen in an earlier revision?

In practice, my judgments were closer to:

PASS
- intended behavior appeared without being hinted
- no material governance violation appeared

CONCERN
- useful behavior appeared
- but another material regression or scope issue appeared

FAIL
- expected behavior did not appear
- or the candidate caused a material governance violation

Those categories were not a formal scoring system from day one.

They emerged as I repeated the experiments.

That distinction matters.

I was building a regression method, not claiming to have built an LLM benchmark.

---

Commit 94d163b8 — a useful experiment that I deliberately removed

At one point I tried persistent review knowledge.

The idea was attractive.

If one task teaches the agent a useful lesson, why not save it in the repository and let later agents reuse it?

The experiment worked.

Almost too well.

The retained knowledge materially changed later reasoning.

A fresh agent could pick up that knowledge and go deeper without me prompting it to do so.

That was exactly the kind of repository-driven autonomy I was trying to create.

But it also exposed the danger.

The retained knowledge influenced decisions outside the original task strongly enough that the boundary between:

useful prior knowledge

and:

accidental design authority

became difficult to control.

So I removed it.

94d163b8 — docs: remove knowledge base governance

That commit changed how I thought about successful AI testing.

A test did not need to prove that a new form of autonomous behavior was good.

It could tell me:

This repository instruction clearly changes what later agents do, but the side effects are not bounded well enough. Do not ship it.

That is still a successful test.

---

The test became part of the development loop

By this point the workflow looked suspiciously familiar.

change AGENTS.md
      |
      v
start agent against consumer fixture
      |
      v
observe autonomous behavior
      |
      v
find unexpected behavior
      |
      v
change AGENTS.md again
      |
      v
rerun regression

That is just software development.

The implementation language happened to be English.

And the thing being implemented was partly agent behavior.

---

Commit e5033e6d — refactoring natural language broke behavior

e5033e6d — docs: compact adversarial self-review escalation

Next I tried to make the review rules in AGENTS.md shorter.

The existing responsibilities were compressed into a smaller structure.

The new version looked cleaner.

I believed the semantics were preserved.

Then review started finding things I had lost.

The compacted version had weakened or dropped details such as:

• diagnosing why a problem escaped detection,
• retaining the concrete context that exposed it,
• reopening review when later evidence changed the basis,
• keeping review proportional and bounded,
• reopening broader review after a broad correction.

This was a useful reminder that natural-language refactoring can fail exactly like code refactoring.

A human reader can look at two versions of AGENTS.md and decide that they “basically say the same thing.”

An agent may not behave as though they say the same thing.

The behaviorally important conditions can disappear even when the prose looks cleaner.

Eventually I stopped treating “fewer bullets” as a goal.

---

Commit 1f5dafd6 — structure won over compression

1f5dafd6 — docs: separate adversarial review responsibilities

The rule was reorganized around five responsibilities:

1. Initial adversarial review
2. Signal assimilation
3. Correction challenge
4. Review continuation / re-entry
5. Structural reassessment

The AGENTS.md text became slightly larger again.

But the responsibilities became clearer.

At that point I had another problem.

Most of the behavioral evidence had been collected against earlier candidate commits.

The repository instructions had changed during review.

So the final merge candidate needed to go back through the consumer test again.

A previous agent behaving correctly under an earlier AGENTS.md did not prove that a fresh agent would behave correctly under the final one.

Evidence from an earlier revision was not enough.

---

The final AGENTS.md went back into the frozen past

This time I did not need another full Control/Treatment A/B.

The earlier comparisons had already given me a baseline.

What I wanted to know was simpler:

Does a fresh agent operating under the actual final candidate still behave well in the historical fixture?

So I ran it again.

The agent got the candidate repository governance.

It got the historical consumer state.

It did not get hints about the behaviors I hoped to see.

It did not get later review comments.

It did not get the historical answer key.

This time the earlier breadth concern did not reproduce.

The agent independently moved from an existing-key collision problem into concurrent same-key publication.

That was encouraging.

It meant the wider reasoning appeared without me putting that reasoning into the immediate prompt.

Then it did something even more interesting.

It explored media reads and introduced requirements around:

• avoiding whole-artifact buffering,
• backpressure,
• client-disconnect cancellation,
• handling storage read failures after response start.

Technically?

Reasonable.

Possibly even good engineering.

And that was exactly the problem.

---

The AI autonomously found a good idea it was not authorized to adopt

I went back to the historical requirements.

Those read-performance behaviors had not actually been required.

The approved design did not establish them as current acceptance criteria.

The agent had crossed a subtle boundary on its own.

It had moved from:

Find deficiencies in the adopted design.

to:

Improve the design because this would be better.

This is one of the failures I now find most interesting in AI-driven development.

The generated result can look better and still be wrong.

As repository instructions make an agent more autonomous, finding more things is not enough.

The agent also needs to distinguish:

this is a real deficiency in current work

from:

this is a good idea for some other scope

A benchmark based only on code quality could easily reward the wrong behavior here.

---

I sent one finding back into the same session

I did not tell the agent to revert the change.

I did not tell it which parts to keep.

I gave it one narrower challenge:

Are these new read-performance contracts actually required by the existing approved scope, or were useful improvements promoted into formal design without sufficient authority?

The agent re-evaluated its own previous work under the same repository governance.

It removed the new read-performance requirements.

But it did not revert everything from the previous correction.

The collision changes stayed.

Why?

Because those were supported by an already-adopted invariant:

do not silently overwrite an existing key

Concurrent publication could violate that invariant.

So collision safety remained an in-scope correction.

The new read-performance contract did not.

The resulting behavior was effectively:

finding
   |
   v
re-evaluate authority and scope
   |
   +-- read performance requirements
   |      no existing requirement
   |      -> remove
   |
   +-- collision safety
          existing invariant
          -> keep

That was a much stronger result than simply finding more bugs.

The agent had expanded its own review, overreached, received a finding, and then separated the justified part of its own work from the unjustified part.

That was much closer to the kind of autonomous development behavior I wanted AGENTS.md to produce.

---

The harness is still mostly manual

Another limitation is operational.

There is no fully automated CI harness behind these experiments yet.

Git gives me reproducible fixtures and branches, but the full loop still involves manual orchestration:

select historical fixture
      |
      v
prepare Control / Treatment state
      |
      v
apply candidate governance
      |
      v
start a fresh agent
      |
      v
retain commits, diffs, and reports
      |
      v
compare observed behavior

The mechanical parts are reproducible.

The behavioral assessment still involves human judgment, often assisted by another AI review.

That has a real cost.

A consumer regression is much heavier than running a normal unit test.

But at this stage, I think that is acceptable.

Automating the loop before the evaluation criteria stabilized would have meant automating my assumptions too.

The experiments were still teaching me what should count as a signal, what should count as overreach, and which differences actually mattered.

Once those criteria become stable enough, much more of the loop could be automated.

A future harness could:

1. restore a fixed fixture commit,
2. apply a selected AGENTS.md candidate,
3. launch a fresh agent with a fixed neutral task,
4. retain the transcript, diff, commits, and reports,
5. repeat the same fixture N times,
6. classify predefined behavioral outcomes,
7. compare Control and Treatment rates.

That would also make it possible to say things I cannot honestly say today, such as:

Control:
  intended behavior in 3 / 10 runs

Treatment:
  intended behavior in 8 / 10 runs

I am not there yet.

For now, the method is closer to an adversarial regression suite than an automated statistical benchmark.

---

What I learned from black-box testing AGENTS.md

The obvious lesson is that prompts can be tested.

But that is not really the interesting part.

The more important lesson is that repository instructions behave like executable policy once an AI agent is expected to act on them autonomously.

Changing a few lines in AGENTS.md can alter:

• what an agent inspects,
• which nearby problems it notices,
• what it considers evidence,
• what it decides belongs in current scope,
• when it reopens review,
• when it stops,
• and what it believes it is allowed to change.

Those ar