DEV Community

Cover image for Why Does Your AI Keep Guessing About Your Project?
Malik Chohra
Malik Chohra

Posted on Originally published at codemeetai.substack.com

Why Does Your AI Keep Guessing About Your Project?

TL;DR

  • Your coding agent guesses whatever your project leaves unsaid: the test command, the folders it may change, which of two prices is current. You pay for each guess in review time.
  • Four small files give most of those answers once, for every agent. The free Second Brain Starter, a plain-Markdown workspace that gives your AI a memory it can continue across sessions, ships the first three files. You add the fourth yourself.

I write Code Meet AI, the newsletter where I document how I actually use AI to ship real products, receipts included. New guides land there first.

If you run Claude Code, Codex or Cursor on a project most days, this is the setup that saves you the same correction every session. The last piece, How to brief an AI agent in six lines, gave every task six questions to answer before an agent starts: context, owner, effects, evidence, continuity and learning. This one moves most of those answers into the project itself, so you write them once.

Your agent forgets everything between sessions

Every session starts from zero. A project that is ready for an agent writes the answers down where the agent looks first: what the project is, how to prove a change works, what is off limits, and where each fact lives.

The method behind this is ICM, the Interpretable Context Methodology: small files point to the next file, so the agent reads only what the task needs. Jake Van Clief and David McDermott wrote it up (arXiv:2603.16021). My own second brain runs on it, and here it moves from a folder of notes to a code project.

Four small files are enough to start

Say you have a small app. It keeps its data in Supabase and takes payments with Stripe. Four files make it agent-ready.

  1. One main file that every coding agent reads first, with the commands that prove a change works.
  2. A one-line file for Claude Code that sends it to the main file.
  3. A facts file, where each price, plan name and setting has one home.
  4. A short file inside the payments folder, with stricter rules for the Stripe code.

Four files appear one by one: the main file, the one-line Claude file that points to it, the facts file, and a stricter file inside the payments folder that wins for edits there

The main file follows an open format called AGENTS.md, "used by over 60k open-source projects" according to its site. Recent versions of Claude Code read it when a project has no CLAUDE.md of its own, and when it has one, a single import line pulls it in (Claude Code docs).

1. Send every agent to the same file first

One front door saves every agent the search. My own second brain's Claude file holds exactly one line, and so does the one in Spec Harness, my open-source kit for code projects:

@AGENTS.md
Enter fullscreen mode Exit fullscreen mode

Anything only Claude needs goes below the import. Say the only Claude habit you want is a planning step before changes to the payments folder. The whole file then reads:

@AGENTS.md

Before you change anything in the payments folder, show me a plan first.
Enter fullscreen mode Exit fullscreen mode

Looking after it takes almost no work. Every rule that all agents need goes in the main file, and the Claude file gains a line only when a habit applies to Claude alone. When it grows past a few lines, move the extra lines into the main file. Keep the main file short. The same Claude Code docs say "target under 200 lines per CLAUDE.md file. Longer files consume more context and reduce adherence."

2. Give each fact one home

Two prices in two files is one price too many. The agent picks one, and you find out which in review. Name the one home, and tell the agent to check any copy against it.

Facts live in docs/facts.md. If you find a value anywhere else, check it against that file and tell me.
Enter fullscreen mode Exit fullscreen mode

3. Fence what breaks quietly

Some parts of a project fail without noise: the payment webhook, the database schema, anything that emails your users. Put a written fence around them, and a rule that says stop and ask. The line below does three jobs. The two names are the places where a mistake costs the most and shows up latest. "Without my yes" makes you the gate. "Stop and show me the plan" makes the agent list what it would change before it touches anything, so you review a plan and skip the diff afterwards.

Never change the payments folder or the database schema without my yes. Stop and show me the plan.
Enter fullscreen mode Exit fullscreen mode

Swap in your own folders, such as auth, migrations or billing settings, and keep the list short, because a long list gets skimmed. Say the agent is fixing a bug and finds a payment helper that looks wrong. With the fence it stops and asks you first.

The agents.md site says how a second file works: "The closest AGENTS.md to the edited file wins." So the file inside the payments folder can be stricter than the main one. One catch for Claude Code. The docs say "By default, Claude reads AGENTS.md only when you have no CLAUDE.md in your working directory or above it", and this project has one. I tested it: with a CLAUDE.md at the root, Claude read a file in the payments folder and never loaded that folder's AGENTS.md. The fix is a one-line CLAUDE.md in the payments folder, and then both files load when Claude works there.

The payments folder holds two files. The first is the stricter rule:

payments/AGENTS.md
Never change a webhook handler or a price ID. Show me the plan and wait for my yes.
Enter fullscreen mode Exit fullscreen mode

The second is the one-line import that makes Claude Code load it:

payments/CLAUDE.md
@AGENTS.md
Enter fullscreen mode Exit fullscreen mode

4. Say what done means

An agent that guesses the test command runs the wrong one, or none, and reports done. Write the exact commands and what done looks like.

Done means: the tests pass and the app starts. Run both commands from the Commands section and paste their last lines.
Enter fullscreen mode Exit fullscreen mode

5. Write down what is in flight

The next session starts blind again. One short file with the work in progress and the open questions lets the next agent pick up where this one stopped.

Before you stop, update docs/now.md: what changed, what is left, what is waiting for me.
Enter fullscreen mode Exit fullscreen mode

6. Turn a repeated miss into a rule

Correcting the agent in chat fixes one session. When the same mistake comes back, write the fix into the file, with the date and the reason. My own global file has one from 3 August: read the project's own rules before you touch it. It started as one line, added after a real violation.

When a mistake repeats, add one dated rule to this file with the reason. Leave one-off mistakes out.
Enter fullscreen mode Exit fullscreen mode

How I run agentic coding: Spec Harness

Spec Harness is what I use for coding with agents. It is spec-driven development wrapped in a harness: the spec comes first, and every step after it leaves a record the next session can read. It is free and open source.

Six roles do the core work, one job each. The repo holds more agent files for other jobs. The split exists because the agent that writes the code should never be the one that checks it.

Six roles in a row, one job each: researcher, planner, implementer, tester, verifier, reviewer, with a FAIL arrow looping from the verifier back to the implementer

The researcher and planner prepare the work, the implementer and tester build and run it, and the reviewer checks style and scope. The verifier matters most. It is a separate agent that did not write the code, it checks the result against the goal, and its only answer is PASS or FAIL. A FAIL sends the work back to the implementer.

The files from this article sit around those roles. Rules only tighten, and a failure does not become policy automatically: when something breaks, the cause is logged first.

Spec Harness adds the same front door, a rules index and a memory folder

The four files cover the first four of the six answers: context, owner, effects and evidence. Continuity and learning need a place to write too, and a growing project needs a map of where everything lives. Spec Harness installs all of that in a code project, using other file names for the same jobs.

Install it and you get the same front door and a rules index that says where each kind of fact lives. It also adds a memory folder whose active file every session reads, and a learning log for misses. Its own AGENTS.md states that it adds no dependency and never edits your application code during install. Every rule stays marked PENDING, which means not yet written from your own code and checked, until that work is done.

your-project/
├── CLAUDE.md               @AGENTS.md, then the startup order
├── AGENTS.md               hard limits, and what done means
├── RULES.md                rules for one folder; the nearest one wins
├── ai_rules/
│   ├── globalRules.md      where each kind of fact lives
│   ├── context_map.md      the real modules, mapped after install
│   └── rules/frequent_rules.md
├── .memory/
│   ├── 40-active.md        what is in flight, read every session
│   ├── 50-progress.md
│   └── 60-decisions.md     one home for each decision
└── learning/NOTES.md       misses, before they become rules
Enter fullscreen mode Exit fullscreen mode

The six answers on the left each draw a line to the file that holds them on the right, and the learning line lands last

One line from the main file it installs is worth copying into any project, harness or not:

Never invent an import, function, type, or component. Confirm it exists (grep/Read/index) before you call it.

Four checks, written before any code

Say you ask for a button that lets users register with Gmail. You first write the goal as checks that are either true or false: tapping the button opens Google sign-in, a new user gets an account, an existing user is signed in, and a cancelled sign-in changes nothing.

  1. The researcher finds where sign-up lives and sees that the auth folder is fenced, so the work will stop and ask. The planner splits the goal into tasks: add the button, connect Google sign-in, create or find the user, handle a cancelled sign-in.

  2. The implementer writes the button with its tests, one task at a time, and the tester runs the suite.

  3. The verifier goes through the four checks. If a cancelled sign-in still creates an account, that check fails and the work returns to the implementer.

  4. The reviewer checks that the change touches sign-up and nothing else, and the cause of any miss goes into the learning log. A failure does not become policy automatically.

Keep the README, and give the agent a shorter file

The fair objection: your README already explains the project. It does, for a person who can ask a colleague when a step is unclear. An agent acts on what it reads first. So its file is shorter and gives orders: run this, never touch that, stop here. Keep both, and let the agent file point to the README for the long story.

These files ask, and a hook enforces

A hook is a small script the tool runs before or after an action. The Claude Code docs put the limit plainly: "To block an action regardless of what Claude decides, use a PreToolUse hook." So write the rule in the file, and if breaking it would cost you money or data, add a hook as well.

A file asks nicely. A hook just says no.

Make one project agent-ready this week

Pick the project you open most often with an agent, then:

  • Write the main file: what the project is, the two commands that prove a change works, and the folders that need your yes.
  • Add the one-line Claude file that imports it.
  • Next time the agent guesses wrong, add one dated line to the file instead of typing the correction again.

The free Second Brain Starter gives you the main file, the Claude import and a facts file for each project. Builder AI OS is the paid layer for later, once one project runs well.

Start of the series: How to scale a Second Brain into a Company Brain.

The full series lives in Code Meet AI, my newsletter. The next piece goes there first.

Top comments (0)