I’ve been experimenting with AI coding agents in real day-to-day work and ran into a recurring problem, I keep repeating the same engineering principles over and over.
Things like validating input, being careful with new dependencies, or respecting certain product constraints. The usual solutions are prompts or rules.
After using both for a while, neither felt right.
- Prompts disappear after each task.
- Rules only trigger in narrow contexts, often tied to specific files or patterns.
- Some principles are personal preferences, not something I want enforced at the project level.
- Others aren’t really “rules” at all, but knowledge about product constraints and past tradeoffs.
That led me to experiment with a separate “memory” layer for AI agents. Not chat history, but small, atomic pieces of knowledge: decisions, constraints, and recurring principles that can be retrieved when relevant.
A few things became obvious once I started using it seriously:
- vague memory leads to vague behavior
- long memory pollutes context
- duplicate entries make retrieval worse
- many issues only show up when you actually depend on the agent daily
AI was great at executing once the context was right. But deciding what should be remembered, what should be rejected, and when predictability matters more than cleverness still required human judgment.
Curious how others are handling this. Are you relying mostly on prompts, rules, or some form of persistent knowledge when working with AI coding agents?
Top comments (1)
This resonates. We ran into the same challenge building tools for AI agents - the agent needs context, but the right type of context matters enormously.
What's worked for us is thinking about memory in layers:
The problem with prompts is they try to do all three at once. Rules are better at #3 but struggle with #2. And most "memory" systems focus on #2 but miss that some things should be hard constraints, not suggestions.
Your point about "vague memory leads to vague behavior" is key. We found that explicit, structured knowledge (decision records, constraint specifications) retrieves better than conversational summaries. The agent can reason about "never add dependencies without approval" more reliably than a fuzzy memory of "we discussed being careful about dependencies."
Curious what format you're using for your memory entries - are they structured or free-form?