Why I Built a Brain for My AI Agents — and What It Taught Me About Memory

Shweta Patel — Sun, 29 Mar 2026 00:57:09 +0000

`---
title: "Why I Built a Brain for My AI Agents — and What It Taught Me About Memory"
published: true
description: "Anamnesis v0.3.0: The cognitive architecture that emerged when a physician stopped treating agent memory as an engineering problem and started treating it as a clinical one."
tags: ai, opensource, agents, architecture
cover_image:

canonical_url:

Anamnesis v0.3.0: The cognitive architecture that emerged when I stopped treating agent memory as an engineering problem and started treating it as a clinical one.

Dr. Shweta Patel, MD, FACOG — Board-certified OB/GYN · U.S. Navy veteran (13 yrs) · CEO, Gaya Wellness · 43 AI-leveraged products

TL;DR: Anamnesis is an open-source 4D strategic memory engine for autonomous AI agents. It stores judgment alongside facts — reasoning, authority, trust weight, and decay conditions. Version 0.3.0 adds four brain-inspired capabilities: attention gating, batch consolidation, context-mode retrieval, and prospective memory triggers. The cognitive parallels weren't planned. Identical problem constraints produced convergent architecture. MIT licensed. Self-hosted. PostgreSQL + pgvector. Deploys in 30 minutes via Docker.

The Problem No One Is Solving

I run an autonomous agent fleet. Forty-three AI-leveraged products across clinical operations, marketing, content production, patient intake, and internal automation. These agents operate around the clock, making thousands of decisions per day.

And every single session started from zero.

Not because the models weren't smart enough. They were brilliant. The problem was memory-context decoupling — the gap between what an agent knows during a session and what it retains between sessions. It's the most under-addressed failure mode in autonomous AI, and it produces agents that are perpetually amnesiac. Smart enough to solve any problem. Incapable of remembering they already solved it yesterday.

So I surveyed the landscape. Mem0 stores facts with vector similarity and offers the broadest ecosystem integration — a legitimate production tool. Zep adds temporal knowledge graphs and excels at tracking how entities and relationships change over time. Letta treats memory like an operating system, with the LLM managing its own tiers. All of them solve retrieval: find the relevant thing.

None of them solve judgment: what matters most right now and why.

I needed my agents to remember that enterprise leads convert at 34% — but also why that fact was stored, who determined its importance, how it ranks against competing priorities, and under what conditions it stops being true. Facts without reasoning are trivia. Agents that accumulate trivia don't get wiser. They get slower.

I Approached This as a Clinician, Not an Engineer

Here's the thing nobody in the AI memory space seems to be saying: memory-context decoupling is a clinical problem, not an engineering one.

I've spent 13 years as a Navy medical officer and over a decade in clinical OB/GYN. When I looked at agent memory loss, I didn't see a retrieval optimization problem. I saw something I recognized from practice.

Patients with early cognitive decline don't lose facts first. They lose the weighting of facts. They remember their daughter's name and their cardiologist's name, but they can't tell you which one to call during chest pain. The facts are intact. The judgment layer — what matters, what's urgent, what to ignore — is what degrades.

That's exactly what I was watching in my agents. They could retrieve relevant memories. They couldn't tell me which ones mattered more than others, or why, or when to stop trusting them.

That reframe changed the entire architecture. Instead of building a better filing cabinet, I built Dimension 4: the strategic reasoning envelope. Every memory in Anamnesis carries not just content, but a full reasoning payload: why it was stored, who said so, how important it is relative to everything else, and the explicit conditions under which it expires.

`python

A normal memory system stores the fact:

memory.add("Enterprise converts at 34% from demo")

Anamnesis stores the judgment:

client.retain(
bank="sales_ops",
content="Enterprise converts at 34% from demo",
reasoning="2x better than SMB. Focus sales energy here.",
authority="explicit", # CEO said this, not inferred
tags=["revenue", "enterprise"],
decay_condition="after:90d", # revisit in 90 days
)
`

That reasoning field and authority hierarchy are what separate Anamnesis from everything else in this space. When an agent calls reflect(), it doesn't get a list of similar facts. It gets a ranked operating directive — weighted by strategic importance, with citations and gap analysis. Not a memory dump. A briefing.

The Four Dimensions of Memory

Anamnesis retrieves across four dimensions simultaneously, fused through reciprocal rank fusion with a strategic weight boost:

Dimension	Operation	What It Finds
Semantic	vector similarity	Conceptually related memories — "lead prioritization" finds "enterprise converts at 34%"
Temporal	time-aware decay	Recent and frequently accessed memories, filtered by recency and access patterns
Relational	entity graph	Connected entities — "API v2 dependencies" finds engineering, enterprise, and pipeline
Strategic	weight + reasoning	Memories ranked by importance with full reasoning envelopes and authority hierarchy

The first three dimensions — semantic, temporal, relational — exist in various forms across the memory landscape. Dimension 4 is the novel contribution. It's the layer that turns retrieval into judgment.

The Cognitive Parallels We Didn't Plan

This is the part that surprised me.

I didn't set out to model the brain. I set out to make my agents stop being stupid between sessions. But when the architecture stabilized and I started looking at what we'd built, the parallels to human cognitive architecture were everywhere — and they were structural, not superficial.

The thesis: identical problem constraints produce convergent architecture. The brain and Anamnesis solve the same fundamental problem — how to store, weight, retrieve, and prune information under resource constraints so that the most important knowledge surfaces at the right time. When the problem is the same, the solutions converge.

Human Memory	Anamnesis	Key Insight
Hippocampal encoding	`retain()`	Neither system stores raw input. Both extract structured features at encoding time.
Amygdala emotional tagging	Strategic envelope	Both tag memories with importance metadata that biases all future retrieval.
Multi-cue reconstruction	4D recall + RRF	Both retrieve using parallel cues weighted toward importance, not just similarity.
Rehearsal consolidation	`reweight()`	Strength is earned through repeated use, not assigned at creation.
Synaptic pruning	`decay_check`	Forgetting is a feature. Both systems actively prune to keep retrieval clean.
Prefrontal executive	`reflect()`	The layer that transforms remembered facts into actionable judgment.
Source monitoring	Authority hierarchy	Who said it determines initial trust level.

The convergence isn't accidental and it isn't marketing. Research from the intersection of neuroscience and AI has been moving in this direction for years. A 2025 paper in Frontiers in Neural Circuits demonstrated that feedforward projections from sensory cortex to hippocampus function as feature-extracting encoders — the brain doesn't store raw input, it stores structured representations. That's exactly what retain() does: fact extraction, entity resolution, embedding generation. We built the same pipeline independently because the problem demands it.

Similarly, a 2025 paper in Neural Networks on the stability-plasticity dilemma proposed AI architectures with dual learning rates and offline consolidation inspired by hippocampal-cortical coordination. They were describing the same pattern we'd already implemented in reweight() and decay_check. Same problem. Same constraints. Convergent solution.

Why this matters: The convergence validates both directions. If the brain's architecture is a good solution to the memory-under-resource-constraints problem, then AI systems facing the same constraints should converge on similar designs. When they do — independently — it's evidence the architecture is right.

What's New in v0.3.0: Four Brain-Inspired Capabilities

Version 0.3.0 completes the cognitive mapping. Each of these four features has a direct parallel to a specific human memory mechanism — and each one solves a specific operational problem I was seeing in production.

🧠 Selective Attention → `retain_gate`

The brain doesn't encode everything that hits the retina. Neither should your agent. retain_gate filters inputs before encoding — rejects exact duplicates, warns on semantically similar content, and prevents the context pollution that degrades retrieval quality over time. In production, my agents were storing 3-4 near-identical versions of the same decision. Now they don't.

💤 Sleep Consolidation → `batch_consolidate`

During sleep, the hippocampus replays the day's experiences, merges related memories, detects contradictions, and promotes patterns to long-term storage. batch_consolidate does the same thing on a cron job: merges duplicates, flags contradictions, promotes recurring patterns to higher strategic weight, and produces a consolidation report. Your agent's overnight maintenance cycle.

🎯 Mood-Congruent Recall → `context_mode`

Humans in crisis recall different memories than humans in planning mode. Same brain, same memories, different retrieval bias. The context_mode parameter lets you specify an operational state — crisis, planning, review — that biases which memories surface and how they're weighted. An agent triaging a production outage needs different recall than one doing quarterly planning.

⏰ Prospective Memory → `trigger_memory`

Prospective memory is the ability to remember to do something in the future: "remind me to check the oven in 20 minutes." trigger_memory creates semantic tripwires — memories that fire when a future query matches their trigger condition. "When someone asks about API v2, surface the enterprise dependency analysis." Your agent doesn't just remember the past. It plants flags for the future.

The Bug That Proved the Architecture

Two weeks before this release, I found a bug that would have been invisible in a simpler system.

The reflect endpoint was producing stale briefings. Not wrong, exactly — but oddly weighted. Old decisions were surfacing above recent ones in contexts where recency should have dominated. I ran a diagnostic using Anamnesis's own entity heat scoring and temporal analysis. The system told me what was wrong with itself.

The root cause: a temporal score inversion. The scoring function was using last_accessed_at instead of created_at for the temporal dimension, which meant that every time reflect retrieved a stale memory, it refreshed the access timestamp, which made the stale memory score higher in the next retrieval. A feedback loop that reinforced staleness.

Let me be clear about why this matters: the system was self-diagnosable. Because Anamnesis stores reasoning alongside facts, and because the strategic envelope includes temporal metadata, the diagnostic tools could trace exactly why a memory was being overweighted. The entity heat map showed the inversion pattern. The authority hierarchy confirmed no one had explicitly boosted these old memories. The decay conditions had been correctly set but were being bypassed by the access-refresh loop.

This is diagnostic reasoning. It's the same process I use in clinical practice — the system reveals its own blind spots when you have the right instrumentation. A flat vector store with no reasoning layer would have silently produced degraded results until someone noticed the outputs were off. Anamnesis caught its own disease.

Where This Goes

The immediate use case is obvious: give your AI agents strategic memory. Stop building agents that start every session from zero. Store the reasoning behind decisions so your agents don't just remember what happened — they remember why it mattered.

But the longer arc is more interesting to me.

The same architecture that gives machines strategic memory can augment human cognition. If you have a patient with early cognitive decline, the facts are still there — it's the weighting and retrieval prioritization that degrades. What if you connected existing human hardware to an intelligent memory layer that maintained the strategic envelopes? Not replacing memory. Augmenting it. Keeping the judgment intact when the biology starts to slip.

That's speculative. But it's why I named this project Anamnesis — the Greek philosophical concept that knowledge is not learned but remembered. The engine gives agents access to judgment they didn't generate themselves, stored by a planning layer that did. And the same principle might eventually give humans access to judgment their own biology is struggling to maintain.

For now, though: it's MIT-licensed, self-hosted, deploys in 30 minutes, and it will make your agents meaningfully smarter between sessions.

🔗 Anamnesis v0.3.0 — Open Source

4D Strategic Memory Engine for Autonomous AI Agents
PostgreSQL + pgvector · FastAPI · Docker · MCP Server · Python SDK

⭐ View on GitHub

MIT License · Self-hosted · Deploys in 30 minutes · Star the repo if this is useful

Intelligence knows the answer. Wisdom knows which answer matters. Anamnesis gives your agents wisdom.

Dr. Shweta Patel is a board-certified OB/GYN, 13-year U.S. Navy veteran, author of The Book of Hormones, and CEO of Gaya Wellness, where she leads 43 AI-leveraged products across clinical and operational automation. She built Anamnesis to solve a problem in her own agent fleet and open-sourced it because the memory gap affects everyone building with autonomous agents.`

DEV Community: Shweta Patel