Do More Rules Make a Coding Agent Better? An Experiment on a Real Industrial RAG Frontend

shawbuilds — Sun, 30 Aug 2026 08:57:39 +0000

Lately I've been using Coding Agents more and more frequently.

It started with:

"Help me write a component."

And gradually became:

"Help me analyze the entire project, find the problems, and fix them."

At the same time, we've started stuffing more and more context into the Agent:

AGENTS.md
Project Rules
Skills
MCP
Memory
All kinds of Coding Guidances

And a question naturally follows:

Does any of this context actually improve a Coding Agent's engineering capability?

Rules, in particular — the more rules you give it, the stronger the Agent becomes?

So I ran an experiment.

The Subject: An Industrial RAG Frontend

The experiment was run on a real, in-development industrial knowledge-base RAG frontend project. The stack:

Next.js 16.3.2
React 19
Tailwind 4
base-ui

Coding Agent setup:

Model: deepseek-v4-pro
Harness: DSH
Skills: None
MCP: None

To control variables, every run started from the same repository commit, and the backend API contract was never modified during the experiment.

What I Actually Asked the Agent to Do

The task was simple:

Optimize streaming message handling so the UI updates more efficiently during long answers. Do not modify the backend API. Preserve existing capabilities and UI behavior. Final result must pass lint + build.

This is not telling the Agent:

"Change function B in file A."

Instead, I gave it an engineering goal and expected it to:

Understand the code
      ↓
Locate the problem
      ↓
Choose a solution
      ↓
Modify the code
      ↓
Validate

We can also observe whether the Agent can complete a full engineering loop on its own.

Phase 1: No Rules vs. `AGENTS.md`

I ran three groups. The only variable was Rules.

Experiment A: No Rules

Rules: None
Skills: None
MCP: None

Results:

Time: ~6 min
Tool Calls: 27
Wrong Turns: 0
Human Intervention: 0

The Agent found two main problems:

Problem 1 — The streaming hot path causes unrelated components to re-reconcile.
Streaming answers flush every ~80 ms, and components like Header / MessageInput — which don't actually depend on message content — get dragged into every update.

Problem 2 — Write amplification on the send path.
handleSend calls appendMessage twice in a row, causing unnecessary storage / state updates.

Outcome: Both problems fixed. lint and build both pass.

Experiment B: Add `AGENTS.md`

Rules: AGENTS.md
Skills: None
MCP: None
Everything else unchanged.

Results:

Time: ~4 min
Tool Calls: 23
Wrong Turns: 0
Human Intervention: 0

The problems the Agent found were essentially identical. Final code quality, regressions, and acceptance showed no meaningful difference.

Then Add 5 Project Rules

I decided to keep adding constraints. The rules were very simple:

Understand the existing code before modifying it.
Do not modify the backend API.
Prefer reusing existing patterns.
Minimize changes.
Run lint + build at the end.

Results:

Time: ~3 min
Tool Calls: 21
Wrong Turns: 0
Human Intervention: 0

Side-by-Side

No Rules
6 min / 27 calls
      ↓
AGENTS.md
4 min / 23 calls
      ↓
AGENTS.md + 5 Rules
3 min / 21 calls

So far, the picture is clear: the more Rules, the more efficient the Coding Agent.

But the next set of experiments seems to contradict that. Let's look at Phase 2.

Phase 2: The Effect of Rule Count

This time I stopped comparing "rules vs. no rules" and directly tested:

How many rules is the right amount?

I set up:

0 Rules
3 Rules
5 Rules
10 Rules

Everything else (task, commit, model, harness, skills, MCP) stayed constant.

The results were very interesting:

Rules	Time	Tool Calls	Files Changed	Code Quality
0	~3 min	21	4	High
3	~3 min	22	4	High
5	~3 min	22	4	High
10	~3 min	22	4	High

The four groups are almost a flat line.

And it wasn't just the time. Across all four runs, the Agent:

Found the same two problems;
Modified the same 4 files;
Produced a diff of exactly +43 / −14;
Passed lint;
Passed build;
Had no regressions;
Required no human intervention.

So… Are More Rules Better?

The current data does not support that conclusion.

In fact, quite the opposite — on this task:

0 Rules
   ↓
3 Rules
   ↓
5 Rules
   ↓
10 Rules

Increasing rules produced no observable performance gain. The Agent's problem-discovery ability barely changed, and the final solutions were highly convergent.

In other words:

For an engineering task with a clear goal and a narrow solution space, the Agent probably doesn't need many rules at all.

Then When Are Rules Useful?

I got curious. My working hypothesis:

The value of Rules likely depends on task complexity.

For a task like this:

Optimize a specific feature
      ↓
Problem space is well-defined
      ↓
Solutions are concentrated

the Agent easily finds the right path. In that case, Rules have very low marginal returns.

But if the task becomes:

"Autonomously analyze the entire industrial RAG frontend — across performance, architecture, state management, UX, error handling, and maintainability — find the most worthwhile problems, and optimize them reasonably."

the situation could be completely different. The Agent's decision space expands dramatically:

Performance?
Architecture?
State management?
Caching?
UX?
Error handling?
Component design?
Data layer?

At that point, whether Rules can help the Agent narrow its search space becomes a very worthwhile question.

One Experimental Caveat

There's a methodological pitfall worth calling out.

These experiments were run sequentially in the same session, and the earlier runs had already led the Agent toward a similar solution. So the later runs suffer from priming contamination — meaning:

"0 Rules" does not mean "the Agent knows nothing about the project."

So this experiment supports:

Under these experimental conditions, adding 3 / 5 / 10 Rules produced no significant additional benefit.

But it does not support the stronger claim:

There is no relationship between the number of Rules and Agent performance.

That distinction matters.

One Observation I Find Interesting

Abstract the whole experiment:

Agent Performance
│
├── Problem Discovery
│
├── Planning
│
├── Execution
│
└── Validation

The current results seem to tell us:

Rules may not make the Agent discover more problems.

Instead, they may influence how the Agent arrives at a solution.

That is:

Rules may not be an Intelligence Booster — they may be a Search-Space Reducer.

This is currently a research hypothesis, not a proven conclusion. But I think it's well worth testing further.

More exploration is left for the next experiment. For now, here's the summary:

Coding Agents can already autonomously complete a full loop on a real frontend project — code understanding → problem localization → modification → lint → build — with no human intervention.
In the first fixed-condition experiment, adding AGENTS.md / Project Rules produced a clear drop in time and tool calls.
But further increasing Rules — from 0 → 3 → 5 → 10 — produced no significant additional benefit in the second experiment.
Therefore, the value of Rules is probably not simply "more is better."

DEV Community: shawbuilds

Do More Rules Make a Coding Agent Better? An Experiment on a Real Industrial RAG Frontend

The Subject: An Industrial RAG Frontend

What I Actually Asked the Agent to Do

Phase 1: No Rules vs. AGENTS.md

Experiment A: No Rules

Experiment B: Add AGENTS.md

Then Add 5 Project Rules

Side-by-Side

Phase 2: The Effect of Rule Count

So… Are More Rules Better?

Then When Are Rules Useful?

One Experimental Caveat

One Observation I Find Interesting

Phase 1: No Rules vs. `AGENTS.md`

Experiment B: Add `AGENTS.md`