AI coding agents are becoming much more useful, but there is one problem I keep running into:
They forget project-specific lessons.
Not because t...
For further actions, you may consider blocking this person and/or reporting abuse
Modeling lessons as a graph solves the storage side, but the piece I keep snagging on is the retrieval trigger. The agent only benefits if it actually queries the graph before it touches
src/generated/graphql/User.ts, which means something has to slip a lookup into the loop at the right moment. You can store the lesson beautifully and still never read it. How are you planning to fire that retrieval, a hook that runs before any edit and matches on the file path, or are you trusting the model to remember to ask?Good point. The current approach depends on target capabilities.
For tools that support hooks, AgentsMesh can use a PostToolUse hook to query related lessons after file/tool activity and keep the agent aware of relevant context.
For targets without hooks, it falls back to a blocking rule that tells the agent to check lessons before working on a new area. It works reasonably well, but it depends on context quality — if the chat is overloaded or compacted, reliability drops.
So my current recommendation is to start a fresh chat for each new feature/topic, so the blocking rule and retrieved lessons stay visible and effective.
I agree with the direction here, especially the part about project-specific lessons becoming part of the project instead of being re-explained in every session.
The only thing I would push on is the word “memory.” It may be too generic. Not all project memory has the same value.
A rule like “use MobX stores for business logic” is different from “the last attempt to move this logic into React components failed because it broke subscription updates.” The first is a convention. The second is an episode with a cost attached to it.
For coding agents, I think the valuable layer is not generic “how to code” memory. The model already has a lot of that compressed into the weights. The valuable layer is project-contingent memory: decisions, failed attempts, local conventions, generated-file boundaries, fragile paths, weird commands, and the reasoning behind past tradeoffs.
So the question is not only “how do we store project memory?” It is also: what kind of memory is worth storing, and when should it be retrieved?
A graph helps a lot with structure, but the hard part is the trigger. If the agent is about to edit a generated file, run a migration, touch payment logic, or repeat a failed fix, memory should act before the mistake, not after another postmortem.
That is where project memory becomes more than notes. It becomes a local warning system for facts the model cannot re-derive from general coding knowledge.
Thanks Marco, I think you’ve named it better than I did.
“Memory” is too broad. What I’m trying to build with AgentsMesh is closer to a project-local warning system: short lessons about things the model cannot reliably infer from general coding knowledge.
So not “remember everything,” but “remember the expensive/local stuff”: failed fixes, generated-file boundaries, fragile commands, architecture decisions, local conventions, and tradeoffs. In the library these lessons are tied to file, command, and keyword triggers, so they can be recalled before the agent edits the wrong thing or repeats a known mistake.
That trigger layer is definitely the hard part, and I agree it’s where the real value is.
Hm. I'm now thinking... What if as a LLM thinks, it flags important tokens in terms of priority. Then in the feedback loop, inject the graph entries for those flags. It's not very token efficient, but given that it's cache hits, it's negligible performance and cloud-cost... But result is if it says Apple is important, whatever contextual memory it has about apple will be injected during the next loop, which creates a waterfall effect where each iterative loop, it drills down deeper into the memory, as it does it's thinking, essentially skipping the whole grep search and file-view functions entirely, which in turn would actually save on total tokens and context size in the end?
A system instruction can work for it, else could use a markov chain to determine the most unique word in each sentence, which tends to be the most relevant... Normally used to predict next word, which means the one that's markov-link is the lowest, would most likely be the point of friction, where a generic statement becomes an exact instruction... In turn, inferring what's relevant and what's boilerplate.
Thats an awesome initiative!
I think that approach of documenting lessons learned are good for the agent.
My opinion and what I have did:
I've been doing this using Cline agentic tool and it could be able to generate a good result!
Tests are also code documentation because they are exercising impl and will improve design decisions and assert that code is working!
Thanks! Totally agree with you.
Tests are probably one of the best forms of documentation because they describe the expected behavior and verify it at the same time.
I also like your point about contracts over implementations. That’s exactly the kind of context I think agents should remember and reuse, not just keep inside one chat session.
My idea with graph-based lessons is to connect these things together: decisions, contracts, tests, code areas, and previous mistakes, so the agent can retrieve the right context before making changes.
Really interesting that you’re already doing something similar with Cline.
The "telling the agent the same thing every session" problem is genuinely annoying. A flat file works until it doesn't.
The staleness problem is the hard one though, a lesson about a command that changed 3 months ago is worse than no lesson. How are you handling that?
Yeah, I agree — staleness is probably the hardest part.
Right now I’m thinking about lessons having metadata like status, lastVerified, confidence, and links to related files/commands. So if a lesson references a command, it shouldn’t just be a text note — it should be connected to the actual command/config it depends on.
The ideal flow would be: if the related command/file changes, the lesson becomes “needs review” instead of staying silently trusted.
I don’t think old lessons should be deleted automatically, but they should lose confidence or require re-validation. Otherwise, as you said, stale memory can be worse than no memory.
BTW, one more thing I forgot to mention: an agent can also deprecate an existing lesson and propose/create a new one if it detects an inconsistency.
For example, if a lesson references an old command but the actual package scripts show a different command now, the agent can mark the old lesson as deprecated and create an updated lesson instead.
So lessons are not meant to be permanent static rules — they can evolve with the project.
The retrieval-trigger question Nazar raised is the one I keep coming back to, but I would push it one floor under: who authors a lesson, and how do you verify the right lesson fired at the right moment? An agent self-proposing a lesson after a failed change is single-party authorship of "what counts as the reusable principle here," which means the same loop state that produced the failure decides what to remember about it.
The forward I would ask is on planted faults: can you produce a known mistake and verify the system retrieves the right lesson at recurrence, separately from the proposed/approved/deprecated lifecycle? Without that, the lessons graph is a write-side artifact.
Yeah, I think this is exactly the right pressure test. The lessons system today has decent write-side mechanics, including capture guardrails, dead-trigger checks, dedupe/upsert, deprecate/supersede, recall telemetry, and strict graph validation. However, it still mostly proves that "the graph is well-formed and recallable," not that "the right lesson fires when the same mistake recurs."
The planted-fault framing is the missing evaluation layer. I'd like to have a separate retrieval QA suite in which we would seed a known lesson and decoys, recreate the recurrence context, and verify that the expected lesson appears before the action. Ideally, it would appear in the top N and with the expected trigger and reason. This should be independent of the proposed, approved, or deprecated lifecycle.
Regarding authorship: I agree that agent self-capture should be treated as a proposal, not a fact. The review boundary can be Git/PR for now, but in the long term, the graph will probably require explicit provenance and approval states. However, approval only answers, "May this lesson enter memory?" Planted-fault evaluations answer the more important operational question: "Will this memory actually protect us at recurrence?"
Therefore, I'd consider this the next significant milestone. It's not about further refining the write side, but rather, creating a recurrence harness with known faults, expected lesson IDs, decoys, top-N assertions, and telemetry-derived precision and recall.
The current-state disclosure (write-side mechanics through graph validation, but proving "graph is well-formed and recallable" rather than "right lesson fires at recurrence") is honest stage-marking that lets the rest of the conversation be about the missing piece rather than relitigating what's shipped.
The "approval answers may-this-enter-memory, planted-fault answers will-this-protect-us-at-recurrence" cut is the part I want to ratify forward. Two separate gates, not one gate with two phases. Conflating them is how lessons systems can quietly become well-formed write-side artifacts that never get verified at the operational layer. Authorship-gate vs operational-protection-gate as typed slots, both required, neither inferable from the other.
Two refinements on the harness spec. First, the planted faults have to be authored outside whatever distribution the graph validator has already inspected, otherwise the "decoy" is a lesson the validator has reasons to recognize and the harness measures graph hygiene rather than retrieval discriminability. Second, precision and recall both have to run in both directions: catch rate on planted-fault recurrence, and false-positive rate on adjacent contexts where the lesson should not fire. The false-positive direction is what your original post named as the core risk and the harness has to make it measurable.
The milestone framing (recurrence harness with known faults, expected lesson IDs, decoys, top-N assertions, telemetry-derived precision and recall) is the right thing to be locking in as next. When the first harness run lands, the precision/recall numbers in both directions will be a useful empirical data point for the lessons space.
Thanks, this is a very good point.