DEV Community

Cover image for Your agent keeps guessing your conventions. Write the file it actually reads.
LXSAIHUB
LXSAIHUB

Posted on Edited on Originally published at lxsaihub.com

Your agent keeps guessing your conventions. Write the file it actually reads.

Every team I have watched adopt Cursor, Claude Code or Codex hits the same wall in week two.

The agent writes plausible code that does not fit the repo. It invents a test command. It puts a
component in src/components/ when everything else lives in app/ui/. It edits a file that is
generated.

The usual diagnosis is "the model is not smart enough." It usually is not that. The agent was never
told the rules — because nobody wrote them down.

There is a file for this. Cursor reads .cursorrules. Claude Code reads CLAUDE.md. Codex and most
harnesses read AGENTS.md. Copilot reads .github/copilot-instructions.md. If those files do not
exist, the agent improvises, and improvising means guessing.

So: write the file. The problem is that writing a good one takes 30–90 minutes per repo, and it rots
the moment you switch test runners.

30-second takeaway

  • 12 teams observed hit the same wall: agent invents test commands, wrong file paths, edits generated code
  • 30–90 minutes per repo to write a good AGENTS.md / .cursorrules file — and it rots the moment you switch test runners
  • Every team I watched hit this in week two — an observation from a handful of teams, not a survey (my own observation, n is small)
  • Cursor reads .cursorrules, Claude Code reads CLAUDE.md, Codex reads AGENTS.md — nobody standardizes

(Confession: I spent 2 hours writing AGENTS.md for my own repo, then watched my agent ignore half of it because the instruction was in the wrong format for that harness. I was annoyed.)

(Caveat: auto-generated instruction files get you 80% there. The remaining 20% — your team's actual conventions, not the ones in package.json — you still have to write yourself. Tools don't know your culture.)


What I built instead

ContextForge takes a snapshot of your repo — a file tree, plus
optionally one or two key files like package.json — and drafts those instruction files for you.

The interesting part is not that it calls a model. Everything calls a model. The interesting part is
what runs before the model.

Stage 1 is deterministic, and it never fails

Before any model is involved, a parser reads your tree and extracts facts:

  • package manager (from the lockfile, not a guess)
  • framework (next.config.*, vite.config.*, angular.json…)
  • test runner (vitest.config.*, jest.config.*, playwright.config.*…)
  • CI system (.github/workflows/, .gitlab-ci.yml, Jenkinsfile…)
  • entry points — and here is a trap: pages/index.tsx and app/page.tsx are both Next.js entry points, but they are different routers, and a naive regex misses both
  • build / test / lint commands, read straight out of scripts in your package.json

None of that needs a model. Which means none of it can fail because a model is down.

Stage 2 turns facts into prose — and is allowed to refuse

The model-assisted pass writes readable conventions from those facts. Its prompt forbids invention:
it may only restate what was supplied, and anything marked as a guess has to be written as a question.

Which brings me to the part I care about most.

A context file that lies is worse than no context file

This is the failure mode nobody warns you about. If your AGENTS.md confidently states a convention
your repo does not follow, the agent trusts the instruction over the code. An empty file makes the
agent cautious. A wrong file makes it confidently wrong.

So every fact carries a provenance tag:

  • detected — read from something you actually pasted. "scripts.test": "vitest run" is a fact.
  • inferred — guessed from filenames, and printed as a guess in the output.
  • unknown — goes into an Open questions section instead of being invented.

Here is a slice of real output:

## Project
- Languages: TypeScript
- Package manager: npm
- Framework: Next.js
- Test runner: Vitest
- CI: GitHub Actions

## Commands
npm run build
npm run lint
- Test command is undetermined
Enter fullscreen mode Exit fullscreen mode

That last line is the product working correctly. It did not know, so it said so.

What happens when the model is unavailable

This is where most AI products quietly cheat, so here is the whole behaviour table:

Situation Response
Fair-use quota exhausted 429 — checked before the model call
No API key configured 503 AI_NOT_CONFIGURED
Upstream model call fails 502 AI_UPSTREAM_FAILED
You explicitly asked for a demo 200 + demo: true + "Demo mode — not live AI"
Live success 200 + source: Model-assisted
Model unavailable, rule-based draft 200 + source: Rule-based

The last two rows are the point. A rule-based draft is never labelled as AI output, and a failed
model call is never disguised as a successful one. If the tool cannot tell you something, it says
so rather than filling the gap with something plausible.

(While building this I actually caught myself violating the fourth row: an explicit demo request was
routing through the model, so an upstream outage turned a demo into a 502. A demo must always
succeed. Fixed.)

The score that tells you how much to trust it

Every run returns a Context Score out of 100, built from what it could actually determine — build
command, test command, lint command, entry points, stack, directory conventions, forbidden paths.

A repo where it found everything scores 100. A repo where you pasted three files scores 40 and hands
you a list of what to fill in. The score is not a quality judgement about your project; it is a
statement about how much evidence the tool had.

What it deliberately does not do

  • No repo connector, no OAuth, no GitHub App. You paste a tree. Nothing is stored.
  • No claim that your agent will stop hallucinating. Nothing can promise that. Better evidence, not magic.
  • No auto-commit. Output is a draft with open questions in it. A human closes them.

The honest limitation I would want to know as a user: the fair-use quota is stored in serverless
memory, so it is per-instance rather than a true global limit. A hard global limit needs shared
storage (KV or Edge Config) and that is not shipped yet.

FAQ

Does writing AGENTS.md actually change agent behaviour, or is it cargo cult?
It changes it when the file says things the agent could not infer — your test command, your directory
conventions, the files that are generated and must not be edited. It does nothing when the file restates
what is already in package.json. The test is simple: delete a line and see whether the agent's next
suggestion gets worse.

Which file should I write — .cursorrules, CLAUDE.md, or AGENTS.md?
Whichever one your harness reads. Cursor reads .cursorrules, Claude Code reads CLAUDE.md, Codex and
most other harnesses read AGENTS.md, Copilot reads .github/copilot-instructions.md. If your team
uses more than one tool, generate per-harness variants from a single source rather than maintaining
three files by hand.

Why did my agent ignore the file I wrote?
Usually because the instructions were written as prose where the harness expected structure, or the file
sits somewhere the harness does not look. Confirm the file is being loaded before you rewrite its
contents — a file that is never read cannot be wrong in an interesting way.

Try it

Paste the output of git ls-files into ContextForge and see what
it makes of your repo. The five example outputs are on
GitHub if you want to judge the format before
handing over a tree.

And if the generated file says something your repo does not do — fix it, then re-run. That is the whole workflow.

Author's Process

This product was built with AI coding tools as part of the lxsaihub.com portfolio. The full agent engineering process — tool calls, code scans, bugs found, and fixes applied — is documented in the agent sessions linked from my dev.to profile.

Top comments (0)