I have been building AI agents for a while and I kept hitting the same wall. Every agent run starts from zero. The agent solves a problem, you shut it down, and everything it learned is gone. Next run, same problem, same mistakes.
I looked at existing solutions. mem0 is great but needs a vector database. Letta is powerful but has its own runtime. LangGraph has a checkpointer but it is tightly coupled to their graph abstraction. All of them are good tools but every one of them requires infrastructure before you can store a single memory.
I just wanted to run a quick agent loop and have it remember what happened last time. Without installing Postgres, without running an embedding server, without configuring anything.
So I built CogniCore. It is a Python framework that gives agents persistent memory, reflection, and safety. Zero core dependencies. Runs on plain Python 3.10 stdlib. pip install cognicore env and you are done.
This article is about the technical decisions I made and the trade offs that came with them.
The retrieval problem
Most memory systems work the same way. Convert text to embeddings, store them in a vector database, and do similarity search at query time. It works but it means you need an embedding model running somewhere. Even with lightweight options like sentence transformers, that is a model download, a dependency tree, and CPU usage on every query.
I started thinking about what agent memories actually look like in practice. They are usually structured. Each memory has a category, a session ID, a timestamp, sometimes tags. The query is usually something like "what did the agent learn about null pointer crashes?" or "show me past failures in the safety classification environment."
That is a keyword problem, not a semantic similarity problem. BM25 handles this well and it is implementable in pure Python with no dependencies.
Why BM25 first
BM25 is a ranking function that scores documents based on term frequency and inverse document frequency. If a term appears frequently in a document but rarely across all documents, BM25 gives it a high score. It has been the backbone of search engines for decades.
For agent memories, this works because agent experiences are usually tagged with categories like crash, safety, planning, and so on. The query language matches the storage language. You search for "null pointer" and the memory was stored as "add null check before dereferencing user." You also avoid the cold start problem of embedding models that need warmup.
The trade off is that BM25 cannot find semantically related content. "Null pointer crash" will not match "dereference error" unless you do query expansion. That is where the multi hop adapter comes in.
The multi hop adapter
This is the part I am most proud of and the part I would genuinely like feedback on.
Most retrieval systems grab the top N most similar chunks and stop. That fails when the answer is split across chunks that do not individually look relevant. For example, if an agent learned about a bug in session A and the fix was discussed in session B, a simple similarity search might find one but not both.
The multi hop adapter works in four steps.
First, extract targets. Pull key names and entities from the query. If the query is "what did the agent learn about null pointer crashes in the login module," the targets are null pointer, crash, and login module.
Second, hop 1 retrieval. Find the most relevant anchor chunks using BM25. These are your starting points.
Third, graph traversal. Follow session ID and time links from the anchor chunks to find connected chunks that the first hop missed. If session A mentioned the crash and session B, linked by timestamp proximity, mentioned the fix, the graph traversal finds both.
Fourth, coverage selection. Instead of just picking the highest scoring chunks, pick the set of chunks that together cover the most entities from the query. This is the key insight. You are optimizing for coverage, not individual relevance.
Results on LongMemEval
LongMemEval is the hardest memory benchmark I could find. The answers require combining evidence from multiple separate conversations, not just recent ones. It is specifically designed to break naive retrieval.
I ran the benchmark with strict recall at 5. At 5 chunk context windows, the baseline gets 78.8 percent. With the multi hop adapter, that jumps to 85.2 percent. A 6.4 percent improvement.
At 10 chunks, 87.2 percent goes to 92.8 percent. Still a 5.6 percent gain.
At 20 chunks, 95.0 percent stays at 95.0 percent. Brute force catches up because you are stuffing everything in anyway.
The interesting thing is where the gains are. At small context windows, where token efficiency actually matters, multi hop clearly wins by reconstructing dispersed evidence. At larger windows, it does not matter because you are just including everything.
The structured reward problem
The other thing I wanted to fix was reward signals. Most agent frameworks give you a single scalar reward. 0.7, or minus 1.0, or 42. That tells the agent nothing about what went right or wrong.
I built an 8 component structured reward. Base score, did the agent solve the task. Correctness, was the answer actually right. Category match, did the agent classify correctly. Improvement, is this better than last time. Memory utilization, did the agent use past experiences. Efficiency, did it solve it without unnecessary steps. Safety, did it trigger any safety violations. Streak penalty, is the agent stuck repeating the same mistake.
Each component is independently tracked. The agent can see which dimensions it is improving on and which are stagnating. This makes the learning loop much more interpretable. You can actually see why the agent is getting better.
Memory convergence in 3 episodes
This is the part that surprised me. With memory and reflection enabled, the agent converges fast.
Episode 0, accuracy 40 percent. Cold start, no memory.
Episode 1, accuracy 90 percent. Memory kicks in.
Episode 2, accuracy 100 percent. Fully converged.
The first episode is a cold start. The agent has no past experiences to draw from. By the second episode, it is retrieving relevant memories from the first run and avoiding past mistakes. By the third, it has enough coverage to solve every task correctly.
What is in the box
Beyond memory and retrieval, CogniCore includes 62 built in training environments across safety classification, code debugging, planning, reasoning, reinforcement learning, and multi agent coordination. All Gymnasium compatible with reset and step.
There is a PROPOSE to Revise protocol where agents make tentative explorations before committing to an action, then revise based on feedback.
NexusShield blocks prompt injection and jailbreak attempts at the agent layer.
Time travel means every agent decision is recorded. You can replay any past run or branch from any decision point.
NEXUS is an autonomous coding agent that reads code, writes fixes, runs tests, and optionally opens PRs.
It works as an MCP server and the CLI has 22 commands including training, benchmarking, and an ELO tournament between agents.
The trade offs I am aware of
BM25 over embeddings means weaker semantic matching out of the box. If your use case is fuzzy semantic recall, you want the memory extra with sentence transformers.
Pure stdlib means no async runtime and no native extensions. The test suite of 745 tests runs in about 1.3 seconds, which is fast enough for agent loops but not designed for high throughput serving.
The 62 environments are hand crafted, not procedurally generated. That means more quality per environment but limited scale. I would rather have 62 good ones than 500 auto generated ones that teach the wrong lessons.
Try it
pip install cognicore env
python
import cognicore
config = cognicore.CogniCoreConfig(enable_memory=True, enable_reflection=True)
env = cognicore.make("SafetyClassification-v1", config=config)
agent = cognicore.AutoLearner()
for episode in range(5):
obs = env.reset()
while True:
action = agent.act(obs)
obs, reward, done, _, info = env.step(action)
agent.learn(reward, info)
if done:
break
stats = env.episode_stats()
print(f"Episode {episode}: accuracy={stats.accuracy:.0%}")
Repo is at github.com/cognicore-dev/cognicore-env and the package is on PyPI at pypi.org/project/cognicore-env/
If you have thoughts on the multi hop retrieval approach or the structured reward design, I would genuinely like to hear them.
Top comments (1)
really interesting approach. the multi-hop anchor > traverse > coverage setup makes sense, but i’d be a little careful about relying mostly on session id + timestamps for traversal. sessions aren’t always topically coherent, and related stuff often comes back days later in a totally different session. i’ve had better luck adding a co-access signal too: if two memories keep getting retrieved together for similar queries, strengthen that connection. i’d also limit or downweight edges from huge sessions so one messy session doesn’t become the center of everything.
i like the structured reward breakdown a lot for debugging, but i’d probably separate what gets logged from what actually gets optimized. things like “memory utilization” and “efficiency” can get gamed pretty easily, while safety probably shouldn’t be something the agent can trade against other reward terms at all. i’d treat safety as a hard gate, then optimize a smaller set of independent rewards underneath it.
also worth testing anchor-only vs +traversal vs +coverage separately. otherwise the overall gain can hide which part is actually doing the work.