I started this project with a question: how could an AI agent keep useful information between conversations without treating its entire chat history as one big search result?
A memory system needs more than a place to save text. It needs a way to decide what to keep, retain where a memory came from, retrieve it later, and avoid silently overwriting newer information. I built Cognitive Memory for AI Agents to explore those pieces in a small, modular Python project.
The current implementation covers memory formation, evidence storage, canonical persistence, a FastAPI interface, and a background “Dream Engine” that analyzes memories and proposes decisions. Some parts of the larger design—especially Qdrant semantic search and FalkorDB graph retrieval—are still future work. I’ll separate what is implemented from what I want to build next.
The code is in the project repository on GitHub.
From an experience to a memory
The API’s orchestration layer connects the main steps. An incoming experience goes through an ingestor, which validates and normalizes it while keeping details such as its source and scope. A candidate builder turns that event into a possible memory. JEV, the project’s decision layer, then evaluates what should happen.
The current deterministic path is intentionally narrow. It can store a candidate, ignore an exact normalized duplicate in the same scope, or mark a low-confidence candidate as uncertain and requiring review. JEV also defines other actions, such as merging or superseding memories, but defining an action in the decision model does not mean the API automatically carries it out.
That distinction matters to me. Similar wording can be a useful clue, but it is not proof that two statements mean the same thing.
SQLite as the canonical record
When a memory is accepted, SQLite stores the canonical memory record and its domain events. The repository writes these transactionally and tracks a version for each memory. Updates include an expected version, so a stale request can be rejected instead of overwriting a newer change.
The way I think about the storage boundary is simple: the canonical record owns the memory’s identity and current state. Search indexes, when added, should help find memories; they should not become competing sources of truth.
Keeping evidence attached
A memory is more useful when I can trace it back to the information that supported it. The evidence registry stores metadata and connects an evidence identifier to the original content. The object store keeps that content by its SHA-256 hash, which also lets the store verify the bytes when they are retrieved.
The orchestrator registers evidence for experiences that proceed to storage, then attaches the resulting evidence identifier to the canonical memory. This gives the system a provenance path to inspect later.
There is a boundary here: evidence and canonical memory use separate storage mechanisms. Their writes are not one distributed transaction, so the current implementation does not guarantee that both stores will always succeed or recover together.
FastAPI as the interface
FastAPI exposes the memory operations under a versioned API. The current service supports creating, listing, fetching, and updating canonical memories, retrieving memory history, and processing incoming experiences through the orchestrator. The health endpoint confirms that the API process responds; it does not prove that every dependency is healthy.
This API is the part an agent or application can call without handling SQLite or the evidence files directly. The repository also documents a local development workflow and test suite. Its README reports 158 passing tests for the recorded Phase 5 run; I have not rerun that suite while preparing this post.
Where Qdrant and FalkorDB fit
I chose Qdrant and FalkorDB for the design because they address different retrieval questions.
Qdrant is intended to provide semantic vector search: finding memories that are relevant to a query even when they use different wording. FalkorDB is intended to represent connections between memories and entities, so a later retrieval step could follow relationships between related facts.
In the architecture I want to build, those stores would be derived indexes. A retrieval request could use semantic search to find starting memories, then use graph relationships to discover connected context, and finally load the canonical records and their evidence references. The canonical store would remain authoritative.
That is the planned direction, not a description of a working end-to-end Qdrant and FalkorDB flow in the current repository. The current source tree does not contain those integrations, so I can’t claim that graph traversal or vector search is already available.
The Dream Engine and its limits
The project also includes a Dream Engine workflow for selecting memories, finding possible duplicates or conflicts, and producing structured consolidation decisions. Its queue and execution reports are persisted. The current discovery logic uses deterministic similarity, shared-entity checks, and heuristics; it is not a complete semantic conflict detector.
Most importantly, the current orchestrator proposes decisions but does not apply them to canonical memories. It reports zero applied mutations. I see this as a useful safety boundary while the decision and review workflow is still evolving.
What I learned
The main lesson so far is that a memory system is not just retrieval. The hard parts include preserving evidence, keeping one canonical record, handling versions safely, and being clear about which decisions the software can actually execute.
I also learned to keep the architecture’s direction separate from the implementation status. Qdrant and FalkorDB are part of the plan, but they need to be integrated and verified before I describe them as working features.
There is still a lot to build: vector indexing, graph relationships and traversal, stronger cross-store recovery, authentication and authorization, and measured retrieval evaluation. For now, this project is a working foundation for exploring how those pieces could fit together.
AI assistance disclosure: I used an AI coding assistant during development and to help organize this write-up. I checked the implementation claims against the repository and have marked unimplemented components as future work.
Top comments (0)