DEV Community

Cover image for 140 PRs to stop my AI coding agent from forgetting my codebase. The fix was one file.
Jeonghun Mun
Jeonghun Mun

Posted on

140 PRs to stop my AI coding agent from forgetting my codebase. The fix was one file.

I'm building a Bannerlord-like strategy game on my own, with AI coding agents doing a lot of the typing.

Every new session went the same way. The agent lost track of how the game was put together, "fixed" code that
was working, and when I pointed it out I got the classic "You're right, my mistake." Then it happened again.

Over about 140 PRs I tried most of the obvious fixes:

  • Feed it my best session. I pasted in the whole chat from the session where everything went right, like a Space Marine gene-seed. Same agent, same mistakes. I had only swapped the code for chat history.
  • Inject best practices and past failures. It turned into a pile of exceptions, and work slowed to a crawl.
  • A report and a handover doc for every task. Keeping the docs up to date became more work than the code, and soon the docs contradicted each other.

What worked in the end was much simpler.

What agents miss

Coding agents follow imports well. Ask one to change a function and it finds the callers, reads the tests and makes
a reasonable edit.

What it can't find is what the team knows but never wrote down next to the code:

  • "amounts in this module are in cents"
  • "the mobile app reads this table; don't drop columns"
  • "run the payment e2e after touching this"
  • "this file is generated; edit the template instead"

That knowledge lives in someone's head, an old Slack thread, or a README nobody opens before editing.

One file, handed out per file

The structure had to be so simple that neither I nor the model has to think about using it. So I turned it into a
small open-source CLI, readfirst.

It keeps that knowledge in .readfirst/map.json. Files are grouped, and each group lists:

  • checks: the commands to run after changing it, and what each one covers
  • risks: the traps a newcomer would fall into
  • rules: promises the code must keep, and what breaks if it doesn't
{
  "id": "src-billing",
  "title": "charging cards",
  "files": ["src/billing/**"],
  "checks": [{ "run": "npm test -- billing", "why": "charges and refunds" }],
  "risks": [{ "text": "amounts are in cents" }]
}
Enter fullscreen mode Exit fullscreen mode

Before changing a file:

npx readfirst read src/billing/charge.ts
Enter fullscreen mode Exit fullscreen mode

It prints only what applies to that file: the rules, risks and checks, plus the tests and callers that reach it
through imports (JS/TS and Python).

npx readfirst read prints the rules, risks, checks, tests and callers for one file

To make an agent use it, add one line to CLAUDE.md, AGENTS.md or .cursorrules:

Before changing a file, run `npx readfirst read <file>` and follow what it says.
Enter fullscreen mode Exit fullscreen mode

Keeping it from going stale

A map that rots is worse than no map. So there's a GitHub Action:

- uses: jeonghun917/readfirst@v0
Enter fullscreen mode Exit fullscreen mode

On every pull request it posts one comment, updated on each push, with the rules, risks and checks for the files that
changed. It also fails the check when the map stops covering the code, for example when a new file isn't in any
group. The map gets fixed in the same PR that made it stale.

The CLI and the Action are MIT licensed, have no runtime dependencies, make no network or model calls, and need no
account.

The first map

The hard part is writing the first map, because the useful parts are exactly the ones nobody wrote down.
npx readfirst init drafts groups from your folders, and npx readfirst prompt gives your own agent a brief to fill in
the rest.

There's also a hosted option. Install a GitHub App, comment /readfirst fill on an issue, and it reads the repository
plus your existing notes (README, CLAUDE.md, docs) and opens the map as a pull request. Each risk lists the file and
line it came from, so you can check it before merging. Here's the map it wrote for a copy of Express:
jeonghun917/express#2. A couple of the risks it found:

res.sendFile throws unless the path is absolute or opts.root is given, and dotfiles are denied by default;
relaxing this exposes files from user-supplied paths

test/acceptance/ runs the examples through the real app, so editing an example can break npm test

The first hosted map per account is free: readfirst-cloud.vercel.app

What I'd like to hear

Is "groups + checks + risks + rules" the right shape for this kind of knowledge? What would you want an agent, or a
new teammate, to see before touching a file in your codebase?

The longer version of the 140-PR story is
here.

Top comments (0)