Stop Correcting Your Coding Agent Twice: How KeepPlain Turns Repeated Mistakes Into Rules

I've been working with coding agents every day for a long time now: mostly Claude Code, a lot of Codex, and sometimes Cursor and Pi. At some point, I noticed that a big part of my day wasn't spent building anything. It was repeating myself.

"Run the tests inside Docker, not on the host."
"Don't touch the generated file, change the source."
"We don't use Prettier in this repo."
"You forgot the migration again."

Every correction worked for that session. Then the session ended, the context was gone, and a week later a fresh agent made the same mistake in a different task, and I typed the same sentence again.

The obvious answer is "put it in CLAUDE.md." I did. Then I had a CLAUDE.md, an AGENTS.md for Codex, Cursor rules, a Pi prompt, and they all slowly drifted apart. Worse, I only remembered to write a rule down maybe one time in five. The other four times the lesson just died in a chat window.

So I built KeepPlain. This is the first post in a series about how it works and the decisions behind it. I'll be honest about what's done, what's tested, and what isn't.

This part covers one question: how a passing remark becomes a rule your agent actually reads, and how you stay in charge of it.

What KeepPlain is, in one paragraph

KeepPlain is a plugin for Claude Code, Codex, Cursor, and Pi, plus a website. The plugin keeps your agent sessions (prompts, tool calls, failures, the moments you stepped in). It cleans secrets and personal paths out of them on your machine and sends them to your private space. Nothing is public unless you publish it. On top of that history sit three things: search over your own past work, a library of other people's published sessions, and rules. This post is about the rules.

The core idea: a rule is a mistake that came back

The first version of rules was naive, and it's worth telling because the mistake is instructive.

Every session gets a short "playbook": what the task was, what worked, and the pitfalls, which are the places where the agent went wrong or I had to step in. Version one took those pitfalls and offered them as rules.

It was noise. A pitfall from one session is usually about that task: a field name, a business rule, a bug that only existed in that code. "Don't forget to filter by status = active in the invoices query" is a great note for that day and a terrible instruction for every future session. And because the model words the same complaint differently every time, real repeats didn't look like repeats, so the useful stuff didn't stand out either.

So the rule I settled on is strict:

One session is never a rule. A rule is a mistake that repeated in at least two separate conversations or tasks, and that is general, not tied to one task.

The server checks that threshold; it doesn't take the model's word for it. If the model proposes something backed by only one conversation, it gets dropped, no matter how convincing it sounds.

"Separate" matters too. Editing a message in the Claude app starts a new CLI session with a copy of the conversation, and early on that made one mistake look like three. Sessions that are really the same conversation or the same task are grouped into one "family" before anything is counted. Two families, minimum.

How suggestions are made

Once you turn on personal rules, a background job does this, at most once a day per person, and only if something new happened since the last run:

1. It takes the pitfalls from about your last 60 sessions.
2. It gives them to a model together with your existing rules and everything you've turned down before.
3. The model merges corrections that are really the same thing, throws out what isn't a rule, and writes a general rule: what goes wrong, what to do instead, which stacks it applies to, and which sessions back it.

Then the server checks every suggestion separately: is the text sane (no links, no secrets, nothing that looks like an injection), are the stacks real, does it have two families backing it, and does it duplicate an existing rule or something you hid.

One detail I like a lot: the model has to account for every single pitfall. Each one ends up either in a suggestion, or as "this is an existing rule seen again", or dropped with a reason: only one session, task-specific, not the agent's fault, not about software, already known. The server also counts how many pitfalls the model skipped silently. That report is shown as one line on the suggestions page, so you can see why something didn't become a rule, not just what did.

Adding a rule is one click, and you stay in charge

On the Suggested page, each rule shows how many separate conversations it came from and links to the exact moments in those sessions. You can:

• Add it as is. Personal rules don't need a review; it's your agent.
• Edit first, then add.
• Hide it. A hidden suggestion is passed back to the model as "turned down", so it doesn't come back next week in different words.

You can also write a rule yourself. Suggestions only come from repeats, but a rule you write by hand is your call, and that's fine. Every change is versioned, and the history page shows a diff of what your rules looked like before and after.

Stacks: which rules go where

A rule about Laravel migrations has no business in a Rust project. So every rule has stacks.

Stacks come from a fixed list of about 80: Laravel, Django, Rails, Next.js, Vue, Go, Rust, .NET, and so on. When the model was free to invent stacks, I got rules for "x (twitter)" and "json", which the plugin could never match to a repository.

The plugin detects your repository's stacks locally, from manifests such as composer.json, package.json, pyproject.toml, Gemfile, go.mod and Cargo.toml. It reads the repository root and its immediate subfolders, so a monorepo with apps/web and apps/api gets the stacks of both.

There's also a special stack called Everywhere. Some mistakes have nothing to do with the language: "make a plan before you start changing things", "ask before deleting files". Those rules go into every session, even in a repository where no stack was detected.

How rules reach the agent

This is the part I rewrote most, and I think it's the most interesting one.

The first approach was the obvious one: at session start, inject all rules for the repository's stacks. It works, but it's wasteful. You open a Laravel project to fix a CSS bug, and the agent reads twelve rules about queues and migrations that have nothing to do with the task.

So now it's two steps.

1. At session start: only Everywhere rules. In Claude Code, it looks like this at the top of the session:

## KeepPlain rules
Where coding agents went wrong before, whatever the task, from the user's own earlier sessions.
Keep to them; this repository's own instructions and the user's requests come first.

### Your rules: Everywhere
- Always before starting a task — First, we draw up an inspection plan

2. With each prompt: only the stack rules that match it. When you send a prompt, the plugin's prompt hook compares it with your stack rules and adds the matching ones under "KeepPlain rules for this task".

Rule matching does not send or store your prompt. It runs on your machine and is deliberately simple:

• words are cut to their first 5 letters, so "queue" and "queues" match;
• each rule has keywords, written when the rule was made, in English and in the language you write prompts in. A keyword scores 2, a word from the rule's text scores 1;
• a rule needs a score of 2 to be added, and at most 6 rules are added per prompt;
• each rule is given once per session. It's already in context, so repeating it is just waste.

In Claude Code and Codex the rules go in through the hooks as extra context. In Pi they're added to the system prompt before the agent starts. Cursor can't add context on a prompt, so it gets all its matching stack rules at session start. A session never gets more than 40 rules in total.

The wording matters: this repository's own instructions and the user's requests come first. A rule is a reminder of where agents went wrong, not an order that beats what you're asking for right now.

An end-to-end example

This is an illustration of the flow, not a trace from the logs. The mistake itself is real: I've hit it while building KeepPlain.

Task one, on a Monday. I ask the agent to check a model's relations in a Laravel app that runs in Docker. It runs docker compose exec app php artisan tinker and gets a permission error: psysh can't write to its config folder. It tries a few things and ends up running the code as a script instead. The session's playbook gets a pitfall: tinker failed in the container because its config folder isn't writable.

Task two, a week later, different feature. Same container, same error. This time the agent sets XDG_CONFIG_HOME=/tmp and moves on. Another pitfall, worded differently.

The daily pass. Two separate tasks, the same mistake, a general cause, nothing about one particular model or query. That's enough for a suggestion:

tinker in Docker fails with a permission error on its config folder
Run it with XDG_CONFIG_HOME=/tmp, e.g. docker compose exec -e XDG_CONFIG_HOME=/tmp app php artisan tinker.
Laravel · from 2 conversations · Monday's moment · last week's moment

I press Add. It's my agent, so there's no review.

Task three. I type "open tinker and check why the invoice total is off". The prompt hook finds "tinker" among the rule's keywords, the score clears the bar, and before the agent's first command its context has:

## KeepPlain rules for this task
- tinker in Docker fails with a permission error on its config folder — Run it with XDG_CONFIG_HOME=/tmp (Laravel)

If I'd asked about CSS in the same repository, that rule wouldn't have shown up at all.

What I can't show yet is the step after that: proof across many sessions that the agent actually follows the rule instead of reading past it. That's what the per-pitfall report and the "last seen" counter are for, and it's the first thing I'll write about once there's enough data.

Rules that stop being useful

Rules files rot. A framework update might fix something that was a real problem a few months ago, and nobody deletes the old line.

So KeepPlain tracks when each rule was last seen. Every time a new session runs into the same mistake, that counts as a sighting, and the daily pass reports sightings of existing rules. If a rule hasn't been seen in at least 10 sessions on its stack, and at least 30 days have passed, the rules page offers to remove it. You can press "Keep" instead.

I want to be clear about the limit of this. No repeats doesn't mean the rule is useless. It may be the reason the mistake stopped repeating. The system can't tell a rule that failed from a rule that worked, so it only suggests a removal and leaves the decision to you. Whether there is a better signal for this is an open question, and I'll come back to it.

"On its stack" matters: if you didn't touch Rust for two months, that says nothing about your Rust rules.

Personal rules and team rules

Everything above is about personal rules. You accept them yourself, with one click and no review, because they go into your own agent.

Team rules are different, because they end up in other people's agents. So a team rule changes only through a proposal, the same way code changes through a pull request. A proposal has a "why", the sessions behind it, and a review, and it's merged by someone the team chose. Owners and admins can always merge, and a repeat of an existing rule is added as evidence without a new review. The details of team modes, limits and permissions are a topic for a separate post.

Team rules reach agents in two ways, and the team picks one. They can be written into AGENTS.md and CLAUDE.md as a normal file change that goes through your normal code review. Or they can be added to every session, the same way personal rules are. Nothing is written into the repository without a person doing it.

In a team repository you get both kinds. When a personal rule and a team rule say the same thing, the plugin says it once, in the team's wording, and tells you where each rule came from:

KeepPlain added 5 rules to this session: Laravel (3 yours, 2 from Acme)

What leaves your machine

Rules are fetched in the background, so the session-start hook never waits for the network. It reads a local cache, and refreshes it in the background if the cache is older than an hour or the repository's stacks have changed.

The rules request contains only: the names of the stacks the plugin found (laravel,vue,php), the repository's owner on GitHub (so it can tell a team repository from your own), and the names of rule blocks already in the repository's files (so nothing is said twice). It doesn't contain code, file paths, the repository name or your prompts. Rules for a session are never written into your repository.

Off switches, from the narrowest:

• keepplain rules off turns it off on this computer (or KEEPPLAIN_RULES=0);
• a switch in Settings turns it off for your whole account, on every machine;
• /keepplain:rules shows what is active right now and why.

What's tested and what isn't

• Personal rules, suggestions, team proposals and review are built and covered by tests on both the site and the plugin.
• Rules at session start are verified live in Claude Code and Codex.
• Prompt-level rules, Codex, Cursor and Pi are wired the same way and covered by tests, but I'm still checking them live. Cursor has a known race with the context it accepts at session start.
• How good the suggestions are on lots of real data is the open question. The per-pitfall report exists so I can check that.

What's next in the series

• Part 2. Capturing sessions from four agents, and cleaning secrets locally before anything leaves your machine.
• Part 3. The public library: /keepplain:lookup, which lets your agent check how others solved a similar task, and /keepplain:use, which takes a published playbook into your project.
• Part 4. Team rules in depth: review modes, limits, and what happens when a rule goes stale.

If you've built something similar, like a skills repo, a CLAUDE.md you maintain by hand, or a cron job that reads transcripts, I'd like to hear how you decide what becomes a rule. That line between a one-off remark and a pattern is the part I keep going back and forth on.

KeepPlain: keepplain.com. It works with Claude Code, Codex, Cursor, and Pi.