DEV Community: Jonathanfarrow

# The 5 memory problems for agents

Jonathanfarrow — Mon, 30 Mar 2026 10:13:37 +0000

You ship an agent. It works well in the demo. Users start using it daily. After a week, someone asks: "Why did you suggest the same thing you suggested on Monday? I told you that didn't work."

Your agent has no answer because it has no memory of Monday. Or worse, it has a memory of Monday but no idea that Monday's approach failed.

This is the problem that shows up in every long-running agent system, and it is not a retrieval problem. Your vector search works fine. Your RAG pipeline returns relevant context. The problem is upstream of retrieval: your agent stores facts but does not learn from outcomes. It records what happened without recording whether it worked.

The research has a name for this gap. Hu et al.'s survey on agent memory identifies three functional categories: factual memory (what the agent knows), experiential memory (how the agent improves from past actions), and working memory (what the agent is thinking about right now). Most production agent systems implement factual memory. Almost none implement experiential memory. And experiential memory is where the leverage is for agents that operate over more than a single session.

the five things that break

After building and watching others build agent memory systems, the same five failure modes keep showing up. They are not obvious at first. They emerge over weeks of usage.

1. the contradiction problem

Your agent stores "Alice lives in London" on day one. On day fourteen, it stores "Alice lives in Berlin." Now there are two facts in the memory store. Which one does retrieval return?

If you are lucky, the more recent one. If you are unlucky, whichever one has a higher similarity score to the current query. If Alice's London fact was stored with richer context and more entity connections, it might actually rank higher than the Berlin fact even though it is stale.

The standard fix is to add a created_at timestamp and bias retrieval toward recency. This helps. But it does not solve the underlying problem: your memory store has no concept of a fact replacing another fact. It has two independent entries that happen to be about the same thing. The relationship between them (one supersedes the other) exists only in the real world, not in your data.

What you actually need is for the system to know that "Alice lives in Berlin" invalidated "Alice lives in London." Not as a heuristic. As a structural property of how the data is stored. The old fact should carry a timestamp marking when it stopped being true. The new fact should be linked to what it replaced. Queries should return the currently valid fact by default, with the full history available when you ask for it.

This is what bi-temporal storage does. Every fact carries a valid_from and valid_until. When a new fact supersedes an old one, the old fact's valid_until gets stamped. The new fact opens with valid_until = None (still active). You do not build this logic in application code. The database handles succession.

2. the "what did we know when" problem

Your agent makes a recommendation on March 10th. On March 15th, new information arrives that changes the picture. A user asks: "Why did you recommend that on March 10th?"

To answer this, your agent needs to reconstruct what it believed on March 10th. Not what is true now. What the system knew then.

This is the difference between valid-time (when was the fact true in the real world?) and transaction-time (when did the database learn about it?). Most memory systems track neither properly. Some track when a fact was stored. Almost none track when a fact became true independently of when it was recorded.

A user might tell your agent on March 15th that they moved to Berlin on March 1st. The valid-time is March 1st. The transaction-time is March 15th. If someone queries the system as of March 10th, the correct answer depends on which clock you are asking about. "What was true on March 10th?" returns Berlin (valid-time). "What did we know on March 10th?" returns London (transaction-time, because the system had not learned about the move yet).

Audit trails, debugging agent decisions, and understanding why your agent behaved the way it did all require both clocks. One timestamp column does not cut it.

3. the "it happened at the same time" problem

Alice changed jobs and cities in the same month. Your agent knows both facts independently. But it has no way to connect them temporally. It cannot answer "did those changes happen at the same time?" because its memory has no concept of temporal intervals, let alone relationships between intervals.

This matters more than it might seem. Temporal co-occurrence is how humans detect patterns: "every time we deploy on Friday, something breaks over the weekend" is a temporal pattern. "The user's engagement dropped when we changed the pricing model" is a temporal correlation. Agents that operate in domains with temporal structure (which is most domains) need to reason about how time periods relate to each other.

Allen's Interval Algebra defines 13 relationships between two time intervals: before, after, meets, overlaps, starts, finishes, equals, and their inverses. If your memory system stores facts with proper valid-time intervals, these relationships become queryable. "Find all cases where one relationship ended exactly as another began" is a query, not an application-level computation.

4. the "every session starts from zero" problem

Your agent debugged a tricky authentication issue last week. It tried three approaches, two failed, one worked. This week, a similar issue comes up. The agent starts from scratch.

This is the experiential memory gap. The agent stored facts about last week's session (what was tried, what error messages appeared). But it never synthesized those facts into a lesson: "For auth issues in this codebase, check the token expiry configuration first. Retry logic is usually a red herring."

The research describes experiential memory at four levels of abstraction:

Case-based: Store the raw trajectory. What happened, step by step. High fidelity but expensive to retrieve and hard to generalize from.

Strategy-based: Distill the trajectory into insights. "When X happens, do Y because Z." Transfers across tasks. This is where the leverage is.

Skill-based: Compile strategies into executable code. The agent writes reusable tools for itself.

Hybrid: Maintain multiple levels simultaneously.

Most systems that attempt experiential memory stop at case-based. They store conversation logs or event histories and hope that retrieval will surface the right past experience when a similar situation arises. This works sometimes. It fails when the current situation is similar in intent but different in surface details, because the embeddings do not match well enough.

What you need is a system that automatically compresses raw experiences into generalized insights. Not "here is what happened last Tuesday" but "here is what we have learned from the last ten Tuesdays."

5. the stale confidence problem

Your agent stored a strategy six months ago: "Use approach X for deployment issues." At the time, it had worked three times out of three. Confidence was high.

Since then, the codebase has changed. Approach X has failed twice. But the original memory still has high confidence because nothing in the system updates confidence based on new outcomes.

This is subtler than the contradiction problem. The fact is not wrong. It is not superseded by a new fact. Its reliability has just degraded over time as conditions changed. Your memory store has no way to express this unless you build outcome tracking into the memory entries themselves.

What you need is for memory strength to adjust based on evidence. When an approach succeeds, the associated memory should strengthen. When it fails, it should weaken. The scoring should handle both early uncertainty (few data points) and distribution shift (what used to work no longer does).

6. the multi-hop problem

A user asks your agent: "What API does our project use that's built by the company Steve used to work at?"

No single fact in the memory store answers this. The answer requires chaining through multiple entries: Steve's employment history leads to the company. The company's products lead to the API. The project's dependencies lead back to the same API. Three hops through connected facts.

Flat memory stores handle this terribly. Vector similarity retrieval looks for entries that match the question. But the question mentions Steve, a company, and an API in the same breath, and no single stored fact connects all three. You can try query expansion, generating multiple related queries and merging results. But expansion can only rephrase what is already in the question. It cannot discover connected entities that exist only in the memory store. If the user does not mention the company by name, query expansion cannot find it.

The research is clear on this: for genuine multi-hop queries where the connecting entities are not in the question, you have two reliable options. Let the agent loop (iterative retrieval, calling the search tool multiple times) or build structural connections (a knowledge graph with traversable edges).

Iterative retrieval works but is slow and unpredictable. The agent has to guess what to search for next. It might take two hops or twelve. It might go in circles. And the more hops required, the more likely the chain breaks.

A knowledge graph solves this structurally. The edges exist. The traversal follows them. "Start at Steve, follow worked_at edges to find the company, follow builds edges to find their products, check which product matches the project's dependencies." That is a graph pattern, not a search query:

MATCH (steve:Person {name: "Steve"})-[:worked_at]->(company)
      -[:builds]->(api)<-[:depends_on]-(project)
RETURN api.name, company.name

One query. Deterministic. No iterative guessing. And because edges in MinnsDB are temporal, you can ask the time-aware version: "What API does our project use that is built by the company Steve currently works at?" versus "...used to work at?" The WHEN clause distinguishes between active and historical edges.

This is why the topology of your memory matters. Flat (1D) memory with vector search handles single-hop factual lookups well. But the questions agents actually need to answer in practice are often multi-hop. "What do we know about the context around this error?" is multi-hop. "What changed in this user's situation recently?" is multi-hop. "Has this approach worked before in similar circumstances?" is multi-hop with a temporal dimension.

The research recommends starting flat and moving to graphs when you observe specific retrieval failures. In our experience, those failures show up faster than you expect. As soon as your agent operates over a domain with relationships between entities (which is most domains), multi-hop queries become the norm, not the exception.

what these problems have in common

All six problems share a root cause: the memory system does not model time and structure as first-class dimensions.

The contradiction problem is a temporal succession problem. The audit problem requires two independent time axes. The co-occurrence problem requires temporal intervals with algebraic relationships. The experience problem requires temporal segmentation and consolidation over time. The confidence problem requires outcome tracking that evolves over time. The multi-hop problem requires structural connections between facts, with temporal awareness of which connections are currently valid.

A key-value store with vector search does not solve any of these. A relational database with a created_at column solves the first one partially and the rest not at all. A knowledge graph without temporal edges solves the multi-hop problem but leaves temporal reasoning to application code.

The insight we built MinnsDB around is that agent memory is fundamentally a temporal and structural problem. Not a storage problem. Not a retrieval problem. If your storage layer understands time and relationships, all six failure modes have clean solutions. If it does not, you end up building increasingly complex application logic to paper over the gaps.

how temporal storage addresses each one

Contradictions. Every edge carries valid_from and valid_until. When "Alice lives in Berlin" is recorded, the system checks the ontology: location:lives_in is a functional property (single-valued). The existing London edge gets its valid_until stamped. The Berlin edge opens as active. No application code. The property type defines the succession behavior.

Audit and reconstruction. Every edge also carries created_at (transaction-time). The WHEN clause filters on valid-time. The AS OF clause filters on transaction-time. Compose them:

MATCH (a:Person)-[r:lives_in]->(city)
WHEN ALL AS OF "2024-03-10T00:00:00Z"
WHERE a.name = "Alice"
RETURN city.name, valid_from(r), valid_until(r)

This reconstructs the database's state as it was known on March 10th, showing all historical validity intervals. You can audit any past decision.

Temporal co-occurrence. Allen's Interval Algebra as query predicates:

MATCH (a)-[r1]->(b), (a)-[r2]->(c)
WHEN ALL
WHERE meets(r1, r2)
RETURN a.name, type(r1), type(r2)

"Find all cases where one fact about a person ended exactly as another began." No application logic. A query.

Experience consolidation. The episode detector segments the event stream into coherent experiences. The consolidation engine compresses episodic memories into semantic memories (generalized patterns) and then into schema memories (reusable mental models). Three deployments that all required a migration step become the insight: "deployments in this repo usually need a migration." This runs automatically with configurable thresholds.

Confidence evolution. Each memory tracks outcome counts. Scoring transitions from a Bayesian prior (stable with few data points) to an exponential moving average (responsive to distribution shift as evidence accumulates). Memory strength adjusts based on whether the associated approaches keep succeeding or start failing.

Multi-hop queries. The knowledge graph makes entity relationships traversable. Variable-length paths (-[*1..3]->) handle cases where you do not know the exact number of hops. Because edges carry temporal validity, you can traverse only currently active relationships, or include historical ones, depending on the query. The graph executor handles this with bounded BFS (capped at 10,000 visited nodes per source) and a 30-second query deadline. Multi-hop questions become graph patterns, not iterative search loops.

the developer experience

We think about this from the perspective of an engineer building an agent. You should not need to build temporal succession logic. You should not need to implement your own versioning system. You should not need to write consolidation pipelines or outcome trackers.

You should write events into the system and query it. The temporal modeling, the succession, the episode detection, the consolidation: those are the database's job.

Events go in through the API. State changes, transactions, conversations, observations. The pipeline extracts claims, builds graph edges, detects episodes, forms memories, and consolidates over time. You query with MinnsQL (graph patterns and table queries in one language) or natural language. You subscribe to live queries and get deltas pushed when things change.

-- What does the agent know right now?
MATCH (a:Person)-[r:lives_in]->(city)
WHERE a.name = "Alice"
RETURN city.name

-- What changed in the last week?
MATCH (a:Person)-[r]->(b)
WHERE CHANGED(r, ago("7d"), now())
RETURN a.name, type(r), b.name, change_type(r, ago("7d"), now())

-- What was true six months ago?
MATCH (a:Person)-[r:works_at]->(company)
WHEN "2024-09-15"
WHERE a.name = "Alice"
RETURN company.name

-- Subscribe to changes in real time
SUBSCRIBE MATCH (a:Agent)-[r:KNOWS]->(b:Agent)
RETURN a.name, b.name, r.confidence

The temporal clauses, the change detection, the subscriptions: these are not features you bolt on. They work because time is built into the storage layer from the ground up.

what this is not

MinnsDB is not a general-purpose database. It does not replace Postgres for your application's primary data store. It is not trying to be a vector database or an LLM framework.

It is a database built specifically for the temporal memory problems that agents face. If your agent needs to store facts that change over time, learn from its own experiences, maintain audit trails of what it knew and when, and react to changes as they happen, those are the problems this solves.

If your agent just needs to store and retrieve static facts, a simpler solution works fine. The complexity of temporal storage pays off when your agent operates over time. When sessions span days and weeks. When facts change, the agent needs to know what changed and when. When raw experience should be compressed into knowledge.

Most agents today do not operate this way. But the ones that will matter in six months do.

Try https://minns.ai, the agentic database built for temporal memory.

Agents need true temporal memory. Well, they now have it.

Jonathanfarrow — Sat, 28 Mar 2026 11:56:48 +0000

Agent memory has gotten good at answering "what does the agent know?" Your agent can store that Alice lives in Berlin and retrieve it later. That part works.

The harder question is when. When did Alice move to Berlin? Where did she live before? Did her job change at the same time? And if you had asked the system last Tuesday, before it learned about the move, what would it have told you?

These are temporal questions. And they come up constantly in any agent that operates over more than a single session. Hu et al.'s 107-page survey on agent memory, StructMemEval's benchmarks, Memori's semantic triple work: the research keeps converging on the same finding. The interesting unsolved problems in agent memory are not about storage or retrieval. They are about how facts change over time, how experience consolidates into knowledge, and how an agent's understanding of the world evolves.

We built MinnsDB to make those temporal problems first-class primitives rather than things you build on top of a general-purpose store.

the ceiling of fact-based memory

Most agent memory today is what the research calls token-level factual memory: discrete text entries stored in a vector database and retrieved by similarity. It works. It handles "what does the agent know?" reasonably well.

But the research identifies two other functional categories that flat stores handle poorly: experiential memory (how does the agent improve from past actions?) and working memory (what should the agent be thinking about right now?). More fundamentally, the dynamics axis, how memory is formed, evolved, and retrieved over time, is where flat stores fall apart entirely.

Consider what happens when your agent learns a new fact that contradicts an old one. Alice moved from London to Berlin. In a flat store, you have two options: overwrite the old entry (destroying history) or keep both (creating contradictions during retrieval). Some systems add a created_at timestamp so you can sort by recency. But recency is not the same as validity. Alice's move was recorded on March 15th but she actually moved on March 1st. Two weeks of queries between those dates should have returned Berlin, but the system did not know that yet.

This is not an edge case. It is the normal state of affairs for any agent that operates over time. Facts change. The world your agent monitors evolves. And the question "what is true?" always has a hidden companion: "true as of when?"

Most approaches to this problem treat time as a column you add to your schema. A created_at field. Maybe an updated_at. If you are careful, a valid_from and valid_until pair that you maintain in application code. This gets you partway there, but every temporal question beyond "what is the most recent value?" becomes a query you have to build yourself: versioning logic, interval overlap checks, snapshot reconstruction, change detection between two points. The database stores your timestamps. It does not reason about them.

The deeper issue is that storage is not memory. You can put facts into any database. The hard part is everything that happens after: detecting when a new fact supersedes an old one, consolidating repeated experiences into generalized knowledge, noticing temporal patterns across facts, and knowing what changed between two points in time without diffing snapshots in application code. Those are memory operations, and they need to be primitives in the system, not things you build on top.

bi-temporal storage as first principle

MinnsDB takes a different position. Every edge in the knowledge graph and every row in a relational table carries two independent time dimensions from the moment it is created. You do not opt into temporal tracking. There is no temporal extension to install. Time is the foundation the rest of the system is built on.

Valid-time records when a fact was true in the real world. Alice lived in London from January 2023 to March 2024. She has lived in Berlin since March 2024.

Transaction-time records when the database learned about it. The London-to-Berlin move was recorded on March 15th. If you query the database as of March 10th, it still tells you London, because that is what it believed at that point.

This distinction matters for the same reason the research keeps coming back to temporal dynamics: agent memory is not a snapshot. It is a timeline. And a timeline needs two clocks, one for the world and one for the observer, to be complete.

-- What is true right now? (default: only active facts)
MATCH (a:Person)-[r:lives_in]->(city)
WHERE a.name = "Alice"
RETURN city.name

-- What was true on a specific date? (valid-time filter)
MATCH (a:Person)-[r:lives_in]->(city)
WHEN "2024-01-15"
WHERE a.name = "Alice"
RETURN city.name

-- What did the database believe on March 10th? (transaction-time filter)
MATCH (a:Person)-[r:lives_in]->(city)
WHEN ALL AS OF "2024-03-10T00:00:00Z"
WHERE a.name = "Alice"
RETURN city.name, valid_from(r), valid_until(r)

The WHEN clause filters on valid-time. AS OF filters on transaction-time. Compose them for full bi-temporal queries. This is compiled into the query plan at the planner stage. It is not a post-filter applied after results are already materialised.

what lives inside an edge

The storage model makes bi-temporality concrete. Every GraphEdge carries:

GraphEdge {
    source, target, edge_type,
    weight: f32,              // relationship strength
    confidence: f32,          // extraction certainty
    created_at: Timestamp,    // transaction-time (nanoseconds)
    updated_at: Timestamp,
    valid_from: Option<Timestamp>,   // when this became true
    valid_until: Option<Timestamp>,  // when this stopped being true (None = still active)
    confidence_history: TCell<f32>,  // how confidence changed over time
    weight_history: TCell<f32>,      // how weight changed over time
    properties: HashMap<String, Value>,
}

When a fact is superseded (Alice moves from London to Berlin), the old edge gets a valid_until timestamp. The new edge is created with valid_until = None. No soft-delete flags. No tombstone records. No application-level version tracking. The database maintains the full timeline as a chain of successor states.

The TCell (temporal cell) deserves attention because it solves a specific problem: tracking how a property changes over time without requiring a separate history table. It is a tiered data structure that adapts to cardinality:

TCell<V>:
    Empty                              // zero allocation for static values
    One(timestamp, value)              // single snapshot (common case)
    Small(Vec<(timestamp, value)>)     // 2-64 entries, binary search
    Large(BTreeMap<timestamp, value>)  // 65+ entries, log(n) lookup

This means you can query the confidence of a relationship at any point in its history:

MATCH (a:Person)-[r:works_with]->(b:Person)
WHEN ALL
WHERE a.name = "Alice"
RETURN b.name, confidence_at(r, ago("30d")), confidence_history(r)

The history lives on the edge itself. Not in a separate audit table. Not in application logs. On the edge.

MinnsQL: one language across graph and tables

MinnsDB does not force a choice between a graph model and a relational model. MinnsQL handles both, and they can reference each other within a single query through NodeRef columns that bridge table rows to graph nodes.

Graph patterns for relationships:

MATCH (a:Person)-[r:KNOWS]->(b:Person)
WHERE r.confidence > 0.8
RETURN a.name, b.name, r.weight
ORDER BY r.weight DESC

Table queries for structured records:

FROM orders
WHERE orders.status = "pending" AND orders.amount > 50.0
RETURN orders.id, orders.customer, orders.amount

Cross-model joins bridging the two:

MATCH (n:Person)
JOIN orders ON orders.node = n
WHERE orders.status = "shipped"
RETURN n.name, orders.amount

Tables support full DDL and DML. Column types include String, Int64, Float64, Bool, Timestamp, Json, and NodeRef. Tables are bi-temporal by default. Every UPDATE creates a new row version and closes the old one. You can query all historical versions with WHEN ALL.

The research makes a strong case that the most useful agent memory topology is planar (2D): entries connected via explicit relationships, enabling multi-hop reasoning. MinnsDB's graph is exactly this. But the research also shows that many queries are simple factual lookups that do not need graph traversal. Tables handle those efficiently. Having both in the same query language means you use the right structure for each kind of data without needing two databases and a sync layer.

inside the query engine

A MinnsQL query goes through three stages: parse, plan, execute.

Planning. The parser produces an AST. The planner converts it into an ExecutionPlan with a sequence of physical operators:

Operator	Purpose
`ScanNodes`	Find graph nodes by label and property filters. Uses type index or concept-name index.
`Expand`	Traverse edges from bound nodes. Single-hop or variable-length BFS (configurable hop range).
`ScanTable`	Full scan or index scan of a relational table.
`JoinTable`	Hash join between tables, or graph-to-table lookup via `NodeRef`.
`IndexScan`	Point or range lookup on a primary key index.
`Filter`	Evaluate boolean expressions against intermediate rows.

Each variable in a MATCH pattern gets a numeric SlotIdx during planning, eliminating string lookups at execution time.

Temporal visibility is compiled into the plan as a TemporalViewport:

ActiveOnly         -- valid_until is None (default)
PointInTime(ts)    -- edges valid at a specific moment
Range(start, end)  -- edges valid during an interval
All                -- full history

Edges outside the viewport are rejected at the scan/expand step. They never enter the intermediate result set. This is critical for performance: a graph with millions of historical edges still executes queries quickly because the executor only materializes what is temporally relevant.

Graph execution stores intermediate results in BindingRow structs with a fixed-width slot array. Queries with 16 or fewer variables (the common case) use inline stack storage. Larger queries spill to heap. Variable-length paths use bounded BFS capped at 10,000 visited nodes. Total intermediate rows are capped at 100,000. A 30-second deadline prevents runaway queries.

Table execution uses a separate path optimised for relational access patterns. Predicate pushdown short-circuits to index lookup when a ScanTable is followed by a primary key equality filter. Tables use 8KB slotted pages with blake3 checksums and a binary row codec with O(1) column access.

Both paths share the same post-processing: projection, aggregation (with implicit GROUP BY), HAVING, ORDER BY, DISTINCT, LIMIT.

temporal algebra

The research on temporal reasoning in agent memory is thin. Most systems treat time as a sort key. MinnsDB treats it as a query algebra.

Allen's Interval Algebra is built into MinnsQL as native predicates:

-- Find where one relationship ended exactly as another began
MATCH (a)-[r1]->(b), (a)-[r2]->(c)
WHEN ALL
WHERE meets(r1, r2)
RETURN a.name, type(r1), type(r2)

The full set: overlap, precedes, meets, covers, starts, finishes, equals. Plus SUCCESSIVE(r1, r2) for edges that are temporally adjacent within a tolerance (default 1 second).

This is what lets an agent answer "did Alice change jobs and cities at the same time?" without application logic. The database expresses temporal relationships between facts as first-class query predicates.

Time-bucketed aggregations:

MATCH (a:Person)-[r:payment]->(b:Person)
WHEN LAST "90d"
RETURN time_bucket("1w", valid_from(r)) AS week,
       sum(r.amount) AS total,
       count(*) AS txn_count
ORDER BY week DESC

Change detection across windows:

MATCH (a:Person)-[r]->(b)
WHERE CHANGED(r, ago("7d"), now())
RETURN a.name, type(r), b.name, change_type(r, ago("7d"), now())

The change_type function returns whether a fact started, ended, or was created during the window. Your agent queries for exactly what shifted in any time range. No diffing. No snapshot comparison. The database knows what changed because it never lost track of what was true before.

reactive subscriptions

The research on agent working memory emphasises that agents should not passively buffer context. They should actively control what enters their attention. MinnsDB's subscription system is the reactive version of this principle: instead of polling for changes, the agent declares what it cares about and the database pushes deltas.

SUBSCRIBE MATCH (a:Agent)-[r:KNOWS]->(b:Agent)
RETURN a.name, b.name, r.confidence

This registers a live query. Every graph mutation produces a DeltaBatch on an internal broadcast channel. The SubscriptionManager checks each delta against each subscription's trigger set: a precompiled set of node type discriminants and edge type strings. If a delta cannot affect a subscription, it is rejected in O(1).

For deltas that pass the trigger check, the system chooses between two maintenance strategies:

Incremental maintenance for simple patterns. Operator states track scan candidates, expanded edges, filter results, and aggregation accumulators. Only the delta-affected portion is reprocessed. Cost is proportional to the change, not the result set.

Structural diffing for complex patterns (variable-length paths, node merges). The full query re-executes and the output is compared against the cached previous result. Only actual insertions and deletions are emitted.

// WebSocket: subscribe to a live query
{"type": "subscribe", "query": "MATCH (a)-[r]->(b) RETURN a.name, type(r), b.name"}

// WebSocket: delta pushed on change (~100ms)
{"type": "update", "inserts": [["Alice", "lives_in", "Berlin"]], "deletes": [["Alice", "lives_in", "London"]]}

The subscription fires when Alice moves. Your agent does not poll. It reacts. This is what the working memory research calls "active control over what enters the context." Subscriptions are the mechanism.

from events to episodes to consolidated memory

This is where MinnsDB diverges most sharply from "give the agent a database" approaches. A database stores what you put in it. MinnsDB has an opinion about how raw events become structured memory.

The research describes a dynamic lifecycle: formation (what to store), evolution (how to maintain it), and retrieval (how to access it). Most memory systems implement retrieval well and punt on formation and evolution entirely. MinnsDB implements the full lifecycle.

formation: the ingestion pipeline

Events enter through the HTTP API. Each event flows through a multi-stage pipeline:

Claims extraction. An LLM cascade runs entity extraction, relationship discovery, and fact production. In parallel, a financial NER extractor handles structured patterns (payer, amount, payee). Claims are deduplicated by a normalised (subject, object, category) tuple and scored for confidence. This is the knowledge distillation strategy the research recommends over raw summarisation, extracting discrete facts rather than compressing everything into a summary.

Graph construction. Claims become temporal edges. Edge behaviour is governed by an OWL/RDFS ontology loaded from Turtle files. Functional properties (like location:lives_in) automatically supersede their predecessors: the old edge gets a valid_until stamp, the new edge opens. Append-only properties (like financial:payment) accumulate without supersession. Symmetric properties create edges in both directions. The ontology is the schema. Adding a new domain is writing a Turtle file, not modifying code.

Episode detection. The EpisodeDetector segments the event stream into coherent episodes. This is the part the research calls experiential memory: not just "what happened" but "what was the experience and what was the outcome." Episodes are cut on four boundary triggers:

Context shift: embedding cosine similarity and goal-set Jaccard distance drop below 0.4
Time gap: more than 1 hour of silence between events
Outcome event: a success or failure after at least 2 events in the episode
Cognitive event: a goal formation or planning event starts a fresh episode

Each episode gets a significance score computed from six weighted signals: goal relevance, causal chain depth, duration, novelty (tracked via decay counters over seen contexts and event types), event-type baseline, and optional manual override. A prediction error score measures surprise, novel situations score higher than familiar ones. This drives what gets consolidated and what fades.

evolution: three-tier consolidation

This is where the "dynamics" axis of the research maps most directly onto MinnsDB's architecture. The system implements a three-tier consolidation pipeline modeled on how biological memory actually works, and more specifically, on the strategy-based experiential memory the research identifies as the most underexplored area for practitioners.

Tier 1: Episodic. Raw episodes with summaries, takeaways, outcomes, strength scores. High fidelity, high cost per entry. The case-based level of experiential memory.

Tier 2: Semantic. When 3+ episodic memories share a goal bucket, they are clustered and synthesized into generalized knowledge. Success and failure rates are aggregated. Individual experiences become patterns. "Three deployments that all required a migration" becomes "deployments usually need a migration step." This is the strategy-based level: distilled insights that transfer across tasks.

Tier 3: Schema. When 3+ semantic memories cluster by embedding similarity (threshold 0.80), they are synthesized into reusable mental models. The highest-compression, most general memories. This is what the research calls schematic patterns: high-level templates applicable to novel situations.

Consolidation runs automatically. Source memories decay by a configurable factor (default 0.3x strength) after consolidation. This is not the naive "forget old stuff" approach the research warns against (which destroys rare but critical long-tail knowledge). It is selective consolidation: the knowledge moves up through the tiers, the raw experience decays but is not deleted. The agent builds understanding from experience without you writing consolidation logic.

retrieval: hybrid search

Retrieval uses BM25 keyword search plus embedding similarity plus reciprocal rank fusion. This is the hybrid approach the research consistently recommends over any single retrieval method. BM25 catches exact matches. Embeddings catch paraphrases. RRF merges rankings without requiring score calibration across methods.

The natural language query pipeline goes further: question to LLM intent classification to graph projection (a dynamic walk, not a precomputed view) to multi-source ranking across claims, memories, and strategies.

concurrency model

Write operations flow through 2 to 8 write lanes, sharded by session_id % num_lanes. This preserves FIFO ordering per session while allowing parallel writes across sessions. Each lane is a bounded channel with backpressure: if full, the caller gets a 503 after 5 seconds. Per-lane metrics track in-flight counts, completions, rejections, and latency percentiles via a ring-buffer tracker.

Read operations acquire permits from a read gate: a tokio semaphore with num_cpus * 2 permits. Each query holds a permit for its duration (RAII release on drop). Latency is tracked per-permit lifecycle.

Writes never block reads. Reads never block each other up to the permit limit. The graph sits behind Arc<RwLock<GraphInference>> with a strict lock ordering protocol that makes deadlocks structurally impossible.

storage

The knowledge graph uses SlotVec<GraphNode>, a dense arena giving O(1) node access by ID. Edges, outgoing adjacency lists, and incoming adjacency lists each have their own SlotVec. This layout keeps graph traversal cache-friendly.

Relational tables use 8KB slotted pages with blake3 checksums. Updates create new row versions and close old ones, so table storage is bi-temporal by default. Indexes are rebuilt in memory from persisted pages on startup.

Everything persists to a single embedded ReDB file with a 256MB page cache. The full state (graph, tables, episodes, memories, strategies, WASM modules, auth keys) lives in one file. Binary export and import for snapshots.

what ships in the binary

One Rust binary. One process. No sidecars. No plugins to assemble.

Bi-temporal knowledge graph with OWL/RDFS ontology-driven edge behavior
Relational table engine with bi-temporal row versioning
MinnsQL parser, planner, graph executor, table executor
Reactive subscription manager with incremental maintenance
Episode detector with novelty-weighted significance scoring
Three-tier memory consolidation (episodic, semantic, schema)
Claims extraction via LLM cascade plus financial NER
Hybrid search (BM25 + embeddings + reciprocal rank fusion)
Natural language query pipeline
WASM agent runtime with instruction metering, 64MB memory cap, permission system, cron scheduling
API key authentication with group-scoped multi-tenancy
60+ HTTP endpoints, REST and WebSocket

get started

git clone https://github.com/minnsdb/minnsdb.git
cargo build --release -p minnsdb-server
./target/release/minnsdb-server

Ingest a state change:

curl -X POST http://localhost:3000/api/events/state-change \
  -H "Content-Type: application/json" \
  -d '{"entity": "Alice", "new_state": "Berlin", "old_state": "London", "category": "location"}'

Query the timeline:

curl -X POST http://localhost:3000/api/query \
  -H "Content-Type: application/json" \
  -d '{"query": "MATCH (a:Person)-[r:lives_in]->(city) WHEN ALL WHERE a.name = \"Alice\" RETURN city.name, valid_from(r), valid_until(r)"}'

Subscribe over WebSocket:

wscat -c ws://localhost:3000/api/subscriptions/ws
> {"type": "subscribe", "query": "MATCH (a)-[r]->(b) RETURN a.name, type(r), b.name"}

The research is clear: the hard problems in agent memory are temporal. Formation, evolution, and consolidation over time. Not storage. Not retrieval. The lifecycle.

MinnsDB is the database that implements the lifecycle. Try it out https://minns.ai/

RAG Was Never Memory. This Is.

Jonathanfarrow — Thu, 12 Mar 2026 14:44:25 +0000

Most AI memory systems are still doing the same thing:

chunk text
embed it
retrieve something vaguely similar later
hope the model sorts it out

That works for document search.

It breaks when memory needs to track people, relationships, constraints, and state changes over time.

If a user says:

my budget is €5,000
actually now it’s €7,000
my daughter has a nut allergy
we’re flying from Manchester

you do not want “similar chunks”.
You want the current state, the connected facts, and the ability to reason across them.

That is the gap between retrieval and memory.

And it is why we built MINNS.

MINNS is a memory system for agents that builds a context graph from conversations and lets you query that graph directly. Not raw text retrieval. Not chunk stuffing. Structured memory that can handle multi-hop reasoning and state changes properly.

The best part is how little setup it takes.

Three lines to get started

import { MinnsClient } from 'minns-sdk';

const client = new MinnsClient({ apiKey: 'your-api-key' });

const answer = await client.query("What's the current trip budget?");

That is the core idea.

Three lines, and you are talking to a memory system designed for actual agent context, not just vector retrieval with better branding.

Why this matters

Most so-called memory layers still behave like search.

They can find a sentence that mentions “budget”.
They can find another sentence that mentions “allergy”.
But they do not naturally understand that:

Lily is the daughter
Lily has the nut allergy
the budget changed
the newer budget supersedes the old one
Manchester is the departure location for this specific trip context

That is where a lot of agent systems quietly fall apart.

They look good in demos.
They become brittle the second memory has to evolve.

MINNS was built to solve that properly.

You ingest conversations, MINNS extracts structured facts, resolves entities, tracks state transitions, and builds a context graph that your agent can query.

So instead of asking an LLM to reconstruct truth from a pile of text, you query memory as memory.

A minimal example

Let’s say we ingest a short travel-planning conversation.

import { MinnsClient } from 'minns-sdk';

const client = new MinnsClient({
  apiKey: process.env.MINNS_API_KEY!,
});

await client.ingestConversations({
  case_id: 'holiday-planning',
  sessions: [
    {
      session_id: 's1',
      topic: 'trip',
      messages: [
        { role: 'user', content: "I'm planning a trip to the Amalfi Coast for my family." },
        { role: 'user', content: 'Budget is about 5000 euros.' },
        { role: 'user', content: 'Actually, make that 7000 euros.' },
        { role: 'user', content: 'My daughter Lily is allergic to nuts.' },
        { role: 'user', content: 'We live in Manchester, so flights from Manchester Airport.' },
      ],
    },
  ],
});

Now query it in natural language:

const budget = await client.query("What's the current budget?");
console.log(budget);

const allergy = await client.query('Does anyone have dietary restrictions?');
console.log(allergy);

const flights = await client.query('Where should they fly from?');
console.log(flights);

That is already a very different developer experience from bolting together embeddings, retrieval, filtering, and prompt gymnastics.

Search specific claims too

Sometimes you want direct fact search rather than a composed answer.

You can do that too:

const claims = await client.searchClaims({
  queryText: 'allergies',
});

console.log(claims);

So you have both levels:

query() for memory-aware answers across the graph
searchClaims() for lower-level fact inspection

That is a much better shape for agent systems.

Your LLM can use memory as a tool instead of pretending a prompt full of retrieved chunks is a memory architecture.

Why we built it this way

We built MINNS because too much of the “AI memory” ecosystem still treats memory as a retrieval problem.

It is not.

Real memory has to deal with:

entity resolution
connected facts
evolving user state
supersession
multi-hop reasoning

If a system cannot handle those reliably, it is not really memory. It is search.

That distinction shows up fast when you benchmark.

MINNS has been benchmarked strongly on multi-hop questions and state-change scenarios, which are exactly the cases where naive retrieval starts to crack. That matters because these are not edge cases. They are the normal shape of user context in any serious agent. A user changes plans, updates constraints, refers to family members indirectly, or asks something that requires multiple connected facts to answer.

That is not exotic. That is Tuesday.

Why this is useful for agents

Once memory is queryable like this, your agent loop gets much simpler.

Instead of:

retrieve chunks
stuff them into the prompt
ask the model to work out what matters
hope it prefers the newer fact to the older one

you can do:

ask memory directly
inspect the returned answer or claims
continue reasoning only if needed

That is a much cleaner architecture.

Your LLM becomes the orchestrator.
MINNS becomes the memory layer.
Each part does its own job.

A simple agent pattern

Here is the shape:

const userQuestion = 'What should I know before booking restaurants?';

const memoryAnswer = await client.query(userQuestion);

console.log(memoryAnswer);

If you are building a ReAct-style agent, query() becomes one of the most useful tools in the loop.

The model sees the question, decides it needs memory, calls query(), gets back context-aware output, then either answers or keeps going.

That is far better than making the model reconstruct reality from raw retrieval results.

The setup is the point

A lot of memory tooling asks developers to accept complexity up front:

ingestion pipelines
retrieval tuning
schema work
orchestration glue
ranking hacks
prompt layering to paper over retrieval flaws

MINNS is opinionated in the opposite direction.

The setup should be simple.
The hard part should happen inside the memory system.

That is why this is such a useful starting point:

import { MinnsClient } from 'minns-sdk';

const client = new MinnsClient({ apiKey: 'your-api-key' });

const answer = await client.query("What's the budget?");

Three lines.

And behind those three lines is a memory system built for the exact problems most agent stacks still struggle with.

Final thought

RAG helped AI systems search text.

It did not solve memory.

If you want agents that can handle real user context, especially multi-hop questions and state changes, you need something better than chunk retrieval.

That is what MINNS is for.

Full quickstart

import { MinnsClient } from 'minns-sdk';

const client = new MinnsClient({
  apiKey: process.env.MINNS_API_KEY!,
});

await client.ingestConversations({
  case_id: 'holiday-planning',
  sessions: [
    {
      session_id: 's1',
      topic: 'trip',
      messages: [
        { role: 'user', content: "I'm planning a trip to the Amalfi Coast for my family." },
        { role: 'user', content: 'Budget is about 5000 euros.' },
        { role: 'user', content: 'Actually, make that 7000 euros.' },
        { role: 'user', content: 'My daughter Lily is allergic to nuts.' },
        { role: 'user', content: 'We live in Manchester, so flights from Manchester Airport.' },
      ],
    },
  ],
});

const budget = await client.query("What's the current budget?");
console.log('Budget:', budget);

const claims = await client.searchClaims({
  queryText: 'allergies',
});
console.log('Claims:', claims);

RAG breaks for persistent user context. Here is a React agent that uses a context graph instead.

Jonathanfarrow — Wed, 11 Mar 2026 15:20:03 +0000

RAG helped LLMs search text. It did not give them memory.

If you've built anything serious with LLMs, you've seen the failure mode. A user tells your system they are allergic to nuts, live in Manchester, and have a budget of five grand. Later, the model either forgets, retrieves conflicting fragments, or guesses.

That is not memory. It is retrieval.

Most teams start with RAG. That makes sense. It is useful, familiar, and easy to bolt on. But RAG is fundamentally a document retrieval pattern. It chunks text, embeds it, and returns similar passages at query time. It can surface relevant information, but it does not actually model entities, relationships, or state changes.

If a user says their budget is €5,000 and later updates it to €7,000, a RAG system will often retrieve both passages and leave the LLM to reconcile them. Sometimes it gets that right. Sometimes it does not.

The next step beyond RAG was agentic memory.

Instead of storing raw text and hoping retrieval is enough, agentic memory systems extract structured knowledge that persists across sessions. They track people, facts, relationships, preferences, and constraints as first-class objects. The system does not just find a sentence mentioning Tom. It knows Tom is Sarah’s husband, Lily has a nut allergy, and the family is flying from Manchester.

But even that is not the full story.

Agentic cognition is what happens when the system does more than store and retrieve structured knowledge. It reasons over it. It combines multiple memory structures, resolves entities, tracks state transitions, and produces answers that are grounded in context rather than copied from stored text.

That is where MINNS sits.

You feed it conversations. It builds a context graph. When you query it with nlq(), it does not just retrieve text. It resolves the entities involved, reasons across the graph, and returns a direct answer from the current state of that context.

In this tutorial, we will build a React-style agent with MINNS using the ReAct pattern.

Not an API wrapper. An actual agent.

One that decides what it needs to know, queries a context graph, observes the result, and loops until it has enough information to answer properly.

What we are building

We are going to build a small TypeScript agent that:

ingests a conversation into MINNS
gives an LLM access to MINNS as a tool
lets the LLM query memory during reasoning
loops until it has enough information to answer

The result is a simple but real agent loop. The model does not just generate a response from a prompt. It thinks, acts, observes, and only then answers.

The ReAct pattern

If you have built LLM agents before, you have probably seen ReAct. It is the standard loop for tool-using agents:

Think: decide what information is needed
Act: call a tool
Observe: inspect the result
Repeat or respond: either continue reasoning or answer

This is what separates an agent from a basic chatbot. The model is not just generating text. It is deciding what it needs, fetching that information, evaluating the result, and then deciding what to do next.

In a RAG-based system, the tool usually returns text chunks.

With MINNS, the tool returns answers grounded in a structured, evolving context.

That is the key difference. The system is not forcing the LLM to reconstruct truth from loosely related passages. It is querying a context graph that already models who, what, how, and when.

Prerequisites

You will need:

Node.js 18+
a MINNS API key
an OpenAI API key
basic TypeScript familiarity

Step 1: Project setup

Create a new project and install the dependencies:

mkdir my-agent && cd my-agent
npm init -y
npm install minns-sdk openai dotenv readline
npm install -D typescript tsx @types/node

Create tsconfig.json:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ESNext",
    "moduleResolution": "bundler",
    "esModuleInterop": true,
    "strict": true,
    "outDir": "dist"
  },
  "include": ["src"]
}

Create .env:

MINNS_API_KEY=your_key_here
OPENAI_API_KEY=your_key_here

Step 2: Set up the clients

Create src/agent.ts:

import { MinnsClient } from 'minns-sdk';
import OpenAI from 'openai';
import dotenv from 'dotenv';
import * as readline from 'readline';

dotenv.config();

const minns = new MinnsClient({
  apiKey: process.env.MINNS_API_KEY!,
});

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY!,
});

We are using two clients:

MINNS for memory and context reasoning
OpenAI for the agent loop itself

The LLM decides what to ask. MINNS provides the structured context.

Step 3: Ingest a conversation

Before the agent can reason over anything, we need memory to exist.

Here we ingest a short travel-planning conversation into MINNS:

async function ingest() {
  await minns.ingestConversations({
    case_id: 'travel-booking-2024',
    sessions: [{
      session_id: 'session-1',
      topic: 'holiday-planning',
      messages: [
        { role: 'user', content: "Hi, I'm planning a trip to the Amalfi Coast for my family." },
        { role: 'assistant', content: "Lovely! When are you thinking of traveling?" },
        { role: 'user', content: "Late June. Budget is about 5000 euros." },
        { role: 'assistant', content: "How many people?" },
        { role: 'user', content: "Me, my husband Tom, and our kids Lily who is 8 and Max who is 5." },
        { role: 'assistant', content: "Any dietary requirements?" },
        { role: 'user', content: "Yes — Lily is allergic to nuts." },
        { role: 'user', content: "Oh and we live in Manchester, so flights from Manchester Airport please." },
      ],
    }],
  });

  console.log('Conversation ingested.');
}

This is where MINNS starts to differ from plain retrieval.

It is not just storing messages. It is extracting structured facts and linking them into a context graph. That means the system can represent things like:

Lily is a child in the travelling group
Lily has a nut allergy
Tom is the user’s husband
the family departs from Manchester
the current trip budget is €5,000

Those are not just words in a chunk. They are connected pieces of state.

Step 4: Define the tools

Now we expose MINNS to the LLM through tools.

We will give the model two capabilities:

nlq to ask MINNS a natural-language question
search_claims to inspect extracted facts directly

const TOOLS = [
  {
    type: 'function' as const,
    function: {
      name: 'nlq',
      description: 'Ask MINNS a question. It reasons over the context graph and returns a direct answer.',
      parameters: {
        type: 'object',
        properties: {
          question: { type: 'string' }
        },
        required: ['question'],
      },
    },
  },
  {
    type: 'function' as const,
    function: {
      name: 'search_claims',
      description: 'Search extracted claims and facts from the conversation.',
      parameters: {
        type: 'object',
        properties: {
          query: { type: 'string' }
        },
        required: ['query'],
      },
    },
  },
];

Next, we implement the bridge between the LLM and MINNS:

async function executeTool(name: string, args: Record<string, string>): Promise<string> {
  if (name === 'nlq') {
    const result = await minns.nlq({ question: args.question });

    return JSON.stringify({
      answer: result.answer,
      confidence: result.confidence,
      entities_resolved: result.entities_resolved,
    });
  }

  if (name === 'search_claims') {
    const results = await minns.searchClaims({
      query_text: args.query,
      top_k: 5,
    });

    return JSON.stringify(results.map(r => ({
      fact: r.claim_text,
      confidence: r.confidence,
      entity: r.subject_entity,
    })));
  }

  return '{}';
}

A few things are happening here:

nlq() gives the model a direct way to query the context graph
searchClaims() gives it a lower-level fact lookup tool
both tool results are returned as JSON so the LLM can reason over them in the next step

This is the important architectural point: the LLM is not scraping meaning from raw message history. It is interacting with a structured memory system.

Step 5: Build the ReAct loop

Now we implement the agent loop itself.

This is where the model gets to think, decide which tool to use, observe the results, and either continue or answer.

const MAX_ITERATIONS = 5;

async function agentLoop() {
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  const history: OpenAI.ChatCompletionMessageParam[] = [];

  console.log('Agent ready. Ask anything about the conversation.\n');

  for await (const question of rl) {
    history.push({ role: 'user', content: question });

    for (let i = 0; i < MAX_ITERATIONS; i++) {
      const response = await openai.chat.completions.create({
        model: 'gpt-4o-mini',
        messages: [
          {
            role: 'system',
            content: `You are a helpful travel agent with access to a structured memory system.
Use the nlq tool to ask MINNS questions.
Use search_claims when you need specific extracted facts.
Only answer the user when you have enough information.
If the answer is incomplete, query again.`,
          },
          ...history,
        ],
        tools: TOOLS,
      });

      const message = response.choices[0].message;
      history.push(message);

      if (!message.tool_calls?.length) {
        console.log(`\nAgent: ${message.content}\n`);
        break;
      }

      for (const toolCall of message.tool_calls) {
        const args = JSON.parse(toolCall.function.arguments);

        console.log(`  [${toolCall.function.name}]: ${args.question || args.query}`);

        const result = await executeTool(toolCall.function.name, args);

        history.push({
          role: 'tool',
          tool_call_id: toolCall.id,
          content: result,
        });
      }
    }
  }
}

This loop is the core of the agent.

On each iteration:

the model sees the user’s question and the previous tool outputs
it decides whether it needs to call a tool
if it does, the tool result is added back into the history
the model thinks again using the new information
once it has enough context, it responds

That is ReAct in practice.

The model might answer in one step for a simple question.

For a harder question, it might:

query dietary constraints
inspect family structure
check ages of the children
then combine all of that into a final answer

The agent decides how many steps it needs.

Step 6: Run the agent

Now wire it together:

async function main() {
  await ingest();
  await agentLoop();
}

main();

Start the script:

npx tsx src/agent.ts

Example session:

Conversation ingested.
Agent ready. Ask anything about the conversation.

> What should I know before booking restaurants?
  [nlq]: Are there any dietary requirements for this trip?
  [search_claims]: children ages

Agent: Lily has a nut allergy, so restaurant choices need to be allergy-safe and staff should be able to confirm ingredients clearly. The family is also travelling with two young children, ages 8 and 5, so family-friendly options would make sense. You should also keep total dining spend in line with the overall trip budget of around €5,000.

> Where are they flying from?
  [nlq]: Where is the family departing from?

Agent: They are flying from Manchester Airport.

Notice what happened in the first example.

The model did not answer immediately. It realised it needed more context, used two tools, inspected the results, and then produced a joined-up response. That is the difference between tool use and simple prompting.

Full file

Here is the full working example in one file:

import { MinnsClient } from 'minns-sdk';
import OpenAI from 'openai';
import dotenv from 'dotenv';
import * as readline from 'readline';

dotenv.config();

const minns = new MinnsClient({
  apiKey: process.env.MINNS_API_KEY!,
});

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY!,
});

const TOOLS = [
  {
    type: 'function' as const,
    function: {
      name: 'nlq',
      description: 'Ask MINNS a question. It reasons over the context graph and returns a direct answer.',
      parameters: {
        type: 'object',
        properties: {
          question: { type: 'string' },
        },
        required: ['question'],
      },
    },
  },
  {
    type: 'function' as const,
    function: {
      name: 'search_claims',
      description: 'Search extracted claims and facts from the conversation.',
      parameters: {
        type: 'object',
        properties: {
          query: { type: 'string' },
        },
        required: ['query'],
      },
    },
  },
];

async function executeTool(name: string, args: Record<string, string>): Promise<string> {
  if (name === 'nlq') {
    const result = await minns.nlq({ question: args.question });

    return JSON.stringify({
      answer: result.answer,
      confidence: result.confidence,
      entities_resolved: result.entities_resolved,
    });
  }

  if (name === 'search_claims') {
    const results = await minns.searchClaims({
      query_text: args.query,
      top_k: 5,
    });

    return JSON.stringify(results.map(r => ({
      fact: r.claim_text,
      confidence: r.confidence,
      entity: r.subject_entity,
    })));
  }

  return '{}';
}

async function ingest() {
  await minns.ingestConversations({
    case_id: 'travel-booking-2024',
    sessions: [{
      session_id: 'session-1',
      topic: 'holiday-planning',
      messages: [
        { role: 'user', content: "Hi, I'm planning a trip to the Amalfi Coast for my family." },
        { role: 'assistant', content: "Lovely! When are you thinking of traveling?" },
        { role: 'user', content: "Late June. Budget is about 5000 euros." },
        { role: 'assistant', content: "How many people?" },
        { role: 'user', content: "Me, my husband Tom, and our kids Lily who is 8 and Max who is 5." },
        { role: 'assistant', content: "Any dietary requirements?" },
        { role: 'user', content: "Yes — Lily is allergic to nuts." },
        { role: 'user', content: "Oh and we live in Manchester, so flights from Manchester Airport please." },
      ],
    }],
  });

  console.log('Conversation ingested.');
}

const MAX_ITERATIONS = 5;

async function agentLoop() {
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  const history: OpenAI.ChatCompletionMessageParam[] = [];

  console.log('Agent ready. Ask anything about the conversation.\n');

  for await (const question of rl) {
    history.push({ role: 'user', content: question });

    for (let i = 0; i < MAX_ITERATIONS; i++) {
      const response = await openai.chat.completions.create({
        model: 'gpt-4o-mini',
        messages: [
          {
            role: 'system',
            content: `You are a helpful travel agent with access to a structured memory system.
Use the nlq tool to ask MINNS questions.
Use search_claims when you need specific extracted facts.
Only answer the user when you have enough information.
If the answer is incomplete, query again.`,
          },
          ...history,
        ],
        tools: TOOLS,
      });

      const message = response.choices[0].message;
      history.push(message);

      if (!message.tool_calls?.length) {
        console.log(`\nAgent: ${message.content}\n`);
        break;
      }

      for (const toolCall of message.tool_calls) {
        const args = JSON.parse(toolCall.function.arguments);

        console.log(`  [${toolCall.function.name}]: ${args.question || args.query}`);

        const result = await executeTool(toolCall.function.name, args);

        history.push({
          role: 'tool',
          tool_call_id: toolCall.id,
          content: result,
        });
      }
    }
  }
}

async function main() {
  await ingest();
  await agentLoop();
}

main();

Why this is different from RAG

A RAG-based agent usually works like this:

embed the query
retrieve top-k chunks
put those chunks into the prompt
ask the LLM to infer the answer

That works well for knowledge-base search. It works less well for evolving user context.

MINNS changes the shape of the tool call.

Instead of returning text passages, it returns answers and facts grounded in a structured memory model. That means the LLM is doing less reconstruction from raw text and more decision-making over stateful context.

That matters when the memory is dynamic.

If the user updates their budget from €5,000 to €7,000, that is not just another sentence in a document. It is a state transition. The system needs to understand that the current value supersedes the old one.

That is the difference between retrieval and cognition.

Where to take this next

This example is intentionally small, but the pattern scales.

From here, you can extend the agent with:

reflection steps between tool calls
goal tracking
multi-turn planning
frontend visibility into reasoning steps
temporal queries over evolving state
structured memory types such as ledgers, preferences, and state machines

At that point, you are no longer building a chatbot with retrieval attached.

You are building an agent that can think over memory.

AI coding for JS Devs

Jonathanfarrow — Wed, 20 Nov 2024 18:13:21 +0000

I am hosting an online session where we will live code and go over how to use the AI algorithm - xgboost - to predict house prices.

This is all with the goal of enabling more JavaScript developers to learn how to use ML/AI algorithms. It will be a fairly quick session so if your interest feel free to join.

https://www.meetup.com/ai-london/events/304597357/?eventOrigin=group_upcoming_events

Thoughts on bluesky Vs X for technical discussions?

Jonathanfarrow — Sun, 17 Nov 2024 17:58:30 +0000

Predicting House Prices with XGBoost in Node.js

Jonathanfarrow — Thu, 14 Nov 2024 06:42:31 +0000

What Is XGBoost?

XGBoost is popular machine learning algorithm that regularly places high in Kaggle and other data science competitions. What sets XGBoost apart is its ability to combine multiple weak models (in this case, decision trees) into a strong one. This is done through a technique called gradient boosting, which helps make the algorithm robust and highly effective for a wide variety of predictive tasks.

How Does XGBoost Work?

XGBoost uses gradient boosting, which means it builds trees sequentially where each tree tries to correct the mistakes of the previous trees. Here's a simplified view of the process:

Make an initial prediction (could be the average of all target values)
Calculate how wrong this prediction was (the error)
Build a decision tree to predict this error
Add this tree's predictions to our running prediction total (but scaled down to prevent overconfidence)
Repeat steps 2-4 many times

For example, if we're predicting house prices:

First tree might predict $200,000
If actual price is $250,000, the error is $50,000
Next tree focuses on predicting this $50,000 error
Final prediction combines all trees' predictions

This process, combined with some clever mathematics and optimizations, makes XGBoost both accurate and fast.

Why XGBoost in Node.js?

While XGBoost is originally implemented as a C++ library, there are bindings available for languages like Python and R, making it accessible to a wide range of developers who typically specialize in data and machine learning.

I recently had a project that had a hard requirement for Node.js, so I saw an opportunity to bridge the gap by writing bindings for Node.js. I hope this helps open up the door to more ML for JavaScript developers.

In this article, we'll take a closer look at how to use XGBoost in your Node.js applications.

Prerequisites

Before getting started, ensure you have:

Linux operating system (current requirement for xgboost_node)
Node.js version 18.0.0 or higher
Basic understanding of machine learning concepts

Installation

Install the XGBoost Node.js bindings using npm:

npm install xgboost_node

Understanding the Data

Before jumping into the code, let's understand what our features represent in the house price prediction example:

// Each feature array represents:
[square_feet, property_age, total_rooms, has_parking, neighborhood_type, is_furnished]

// Example:
[1200,       8,            10,           0,           1,                1        ]

Here's what each feature means:

square_feet: The size of the property (e.g., 1200 sq ft)
property_age: Age of the property in years (e.g., 8 years)
total_rooms: Total number of rooms (e.g., 10 rooms)
has_parking: Binary (0 = no parking, 1 = has parking)
neighborhood_type: Category (1 = residential, 2 = commercial area)
is_furnished: Binary (0 = unfurnished, 1 = furnished)

And the corresponding labels array contains house prices in thousands (e.g., 250 means $250,000).

Transforming Your Data

If you have raw data in a different format, here's how to transform it for XGBoost:

// Let's say you have data in this format:
const rawHouses = [
    {
        address: "123 Main St",
        sqft: 1200,
        yearBuilt: 2015,
        rooms: 10,
        parking: "Yes",
        neighborhood: "Residential",
        furnished: true,
        price: 250000
    },
    // ... more houses
];

// Transform it to XGBoost format:
const features = rawHouses.map(house => [
    house.sqft,
    new Date().getFullYear() - house.yearBuilt,  // Convert year built to age
    house.rooms,
    house.parking === "Yes" ? 1 : 0,             // Convert Yes/No to 1/0
    house.neighborhood === "Residential" ? 1 : 2, // Convert category to number
    house.furnished ? 1 : 0                       // Convert boolean to 1/0
]);

const labels = rawHouses.map(house => house.price / 1000); // Convert price to thousands

Training Your First Model

Here's a complete example that shows how to train a model and make predictions:

import xgboost from 'xgboost_node';

async function test() {
    const features = [
        [1200, 8, 10, 0, 1, 1],
        [800, 14, 15, 1, 2, 0],
        [1200, 8, 10, 0, 1, 1],
        [1200, 8, 10, 0, 1, 1],
        [1200, 8, 10, 0, 1, 1],
        [800, 14, 15, 1, 2, 0],
        [1200, 8, 10, 0, 1, 1],
        [1200, 8, 10, 0, 1, 1],
    ];
    const labels = [250, 180, 250, 180, 250, 180, 250, 180];

    const params = {
        max_depth: 3,
        eta: 0.3,
        objective: 'reg:squarederror',
        eval_metric: 'rmse',
        nthread: 4,
        num_round: 100,
        min_child_weight: 1,
        subsample: 0.8,
        colsample_bytree: 0.8,
    };

    try {
        await xgboost.train(features, labels, params);
        const predictions = await xgboost.predict([[1000, 0, 1, 0, 1, 1], [800, 0, 1, 0, 1, 1]]);
        console.log('Predicted value:', predictions[0]);
    } catch (error) {
        console.error('Error:', error);
    }
}

test();

The example above shows how to:

Set up training data with features and labels
Configure XGBoost parameters for training
Train the model
Make predictions on new data

Model Management

XGBoost provides straightforward methods for saving and loading models:

// Save model after training
await xgboost.saveModel('model.xgb');

// Load model for predictions
await xgboost.loadModel('model.xgb');

Further Considerations

You may have noticed there are parameters for this model. I would advise looking into XGBoost documentation to understand how to tune and choose your parameters. Here's what some of these parameters are trying to achieve:

const params = {
    max_depth: 3,              // Controls how deep each tree can grow
    eta: 0.3,                 // Learning rate - how much we adjust for each tree
    objective: 'reg:squarederror',  // For regression problems
    eval_metric: 'rmse',      // How we measure prediction errors
    nthread: 4,               // Number of parallel processing threads
    num_round: 100,           // Number of trees to build
    min_child_weight: 1,      // Minimum amount of data in a leaf
    subsample: 0.8,           // Fraction of data to use in each tree
    colsample_bytree: 0.8,    // Fraction of features to consider for each tree
};

These parameters significantly impact your model's performance and behavior. For example:

Lower max_depth helps prevent overfitting but might underfit if too low
Lower eta means slower learning but can lead to better generalization
Higher num_round means more trees, which can improve accuracy but increases training time

Conclusion

This guide provides a starting point for using XGBoost in Node.js. For production use, I recommend:

Understanding and tuning the XGBoost parameters for your specific use case
Implementing proper cross-validation to evaluate your model
Testing with different data scenarios to ensure robustness
Monitoring model performance in production

Jonathan Farrow

@farrow_jonny

Primer on Recursion

Jonathanfarrow — Thu, 20 Jan 2022 13:06:40 +0000

Introduction

Recursion can be a topic that people hear about and not fully understand or fall into some pitfalls when trying to implement. What I would like to to do is distil some information about recursion that might help further some peoples understanding.

Recursion

Recursion is when a function calls itself with in its body -self invoking.

The example bellow shows the exampleRecursion function calling itself and passing in the new value n. The problem with this function is it will keep calling itself indefinitely until it runs out of stack frames leading to a stack overflow error.

const exampleRecursion = (n)=>{
n = n+1
exampleRecursion(n)
}

The stack is the part of memory where executable is added and operates on a last in last out policy.

Every-time a function is called it is added to the top of the stack then each line inside the function is executed and if another function is called within that function then it is added to the top of the stack to be executed.

const function1 = ()=> {
   // Some code here
   function2();
   // Some code here
 Return "function 1 finished"
}
const function2 = ()=> {
   return "finished";
}

// Invoke the first function
function1();

In the example code above the order of stack execution would be as follows:

The first function1 is added to the stack and then each line of its code is executed.

When it reaches invoking function2 then function2 is added to the top of the stack and its lines of code are then executed.

When function 2 has finished being executed it is removed from the top of the stack and then the rest of function1 lines of code are finished being executed.

Now returning to the problem with recursion is if there is no break clause within function it will keep adding to the stack. To fix this in the first example we can add the break clause to stop at n=10

const exampleRecursion = (n)=>{
if (n=10){
return n
}
n = n+1
exampleRecursion(n)
}

Primitive recursion

A recursive function is primitive when the same functionality can be achieved using loops. For our example we could re design our exampleRecursion function as a for loop.

for (let n = 0; n < 10; n++) {

}

In this example it is more efficient in terms of space to write the function as a for loop as the for loop only adds 1 stack frame.

Efficiency

Recursion can be used to write very simple code as you just need to write a single function that invokes it self. Though these implementations can be very in-efficient. Take for example the this Fibonacci sequence generator


const FibRecursion = (n)=>{

    if (n=== 1){
    return n
    }
    if (n=== 0){
        return n
        }

        return FibRecursion(n-2) + FibRecursion(n-1)
    }

    FibRecursion(5)

To work out the big O complexity of this recursive function we can use the formula 0(bᵈ) where b is the branching factor and d is the depth.

The function would produce this call tree that has depth of 5 and a branching factor of 2. The complexity would be 0(2ⁿ)

If we wrote this function out using for loop iteration. This function would have a complexity of O(n) as we have a single loop of size n.

const fibIterator = (n)=>{
    let fib = [0, 1];


  for(let i = 2; i <= n; i++) {
    fib[i] = fib[i - 1] + fib[i - 2]; 
  }

  return fib;
}

In the next post I will cover tail recursion and using memory functions to improve performance.

Bubble sort

Jonathanfarrow — Mon, 17 Jan 2022 16:56:04 +0000

Bubble sort algorithm

Bubble sort will sort an array by passing through the array and swapping the adjacent elements. It will keep passing through the array until all elements are in order.
Think of it as moving along the array and always asking is my current number bigger than the next number and if it is then it will swap them but it can only swap in the same position twice in each pass.

Notice bellow how the numbers are being ordered with each loop iteration and a single pass is 6 loop iterations for this array size.

How to implement bubble sort

First we will have a do while loop that keeps looping while the array is un sorted.

let keepSwopping;
    do{
        keepSwopping = false;
 }while(keepSwopping)`

Secondly we have a for loop within the do while that will pass through the array.

for (let i = 0; i < length; i++) {
if (array[i] > array[i + 1]) {
 let temp = array[i];
 array[i] = array[i + 1];
 array[i + 1] = temp;
 keepSwopping = true;
                console.log(array);

            }
        }

Thirdly within the for loop we have an If statement that makes a check to see if the current element is larger than the next element in the array.

If it is larger we store the current value in temp. Then we assign the value of the next element[i+1] in the current element[i] and the value of temp in [i+1]. The value of keepSwopping is set true to keep the do while looping.

 if (array[i] > array[i + 1]) {
 let temp = array[i];
 array[i] = array[i + 1];
 array[i + 1] = temp;
 keepSwopping = true;

const unSorted = [3,6,4,7,1,5]
const bubble = (array) => {
    console.log(array)
    length = array.length;
    let keepSwopping;
    do{
        keepSwopping = false;
        for (let i = 0; i < length; i++) {
            if (array[i] > array[i + 1]) {
                let temp = array[i];
                array[i] = array[i + 1];
                array[i + 1] = temp;
                keepSwopping = true;
                console.log(array);

            }
        }
    }while(keepSwopping)
}

bubble(unSorted)

Best and worst case run time

O(n) The best case is when array has already been sorted so we only pass through the loop for the size of the array once.

O(n² ) The worst case is when we have to loop through the array for every item.

O(1) Auxiliary Space