DEV Community: Mark Gyles

SurrealDB Cloud is now available in Frankfurt, Germany

Mark Gyles — Mon, 07 Sep 2026 10:11:28 +0000

SurrealDB Cloud can now be deployed in Frankfurt, Germany. The new region is live today - pick it from the region list when you create an instance.

For teams across the DACH region and continental Europe, this is a second European option and, for most of them, the closer one. Until now the nearest place to run a managed SurrealDB instance was Ireland, on the far western edge of the continent.

Latency and data residency for your workloads

Running in Frankfurt puts the database closer to your users across continental Europe, next to the interconnect that most European traffic already passes through. It also keeps German and EU customer data stored and processed in Germany, satisfying data-residency requirements that previously ruled out a managed deployment. All this while maintaining the same SOC 2 Type 2 controls, ISO 27001, GDPR, encryption and network isolation options as every other region. Agents gain twice over: the region removes the distance, and SurrealDB removes the fan-out. Vector search, graph traversal, full-text search and document reads all resolve in one query against one store rather than separate calls to a vector database, a search engine and a primary database.

Where SurrealDB Cloud runs

Frankfurt joins six other regions:

• US East (N. Virginia)
• US East (Ohio)
• US West (Oregon)
• Europe (Ireland)
• Europe (Frankfurt)
• Asia Pacific (Mumbai)
• South America (São Paulo)

Get started

Create an instance in SurrealDB Cloud and select Europe (Germany) as the region. Everything else - tiers, instance types, versions, backups - works exactly as it does elsewhere, so existing automation needs nothing more than the new region value.

If you are running SurrealDB in production and want high availability alongside in-region deployment, the Scale tier gives you multi-node, multi-availability-zone clusters.

SurrealDB Cloud is now available in São Paulo, Brazil

Mark Gyles — Mon, 07 Sep 2026 09:51:34 +0000

Author: Ignacio Paz

SurrealDB Cloud can now be deployed in São Paulo, Brazil. The new region is live today - pick it from the region list when you create an instance.

For teams building in Brazil and across Latin America, this closes the last big gap in the deployment story: until now the nearest place to run a managed SurrealDB instance was North America, and every query paid for the trip.

Latency and data residency for your workloads

Running in São Paulo puts the database closer to your customers for our South American users. It also keeps Brazilian customer data stored and processed in Brazil, satisfying data-residency requirements that previously ruled out a managed deployment. All this while maintaining the same SOC 2 Type 2 controls, ISO 27001, GDPR, encryption and network isolation options as every other region. Agents gain twice over: the region removes the distance, and SurrealDB removes the fan-out. Vector search, graph traversal, full-text search and document reads all resolve in one query against one store rather than separate calls to a vector database, a search engine and a primary database.

Where SurrealDB Cloud runs

São Paulo joins five other regions:

• US East (N. Virginia)
• US East (Ohio)
• US West (Oregon)
• Europe (Ireland)
• Asia Pacific (Mumbai)
• South America (São Paulo)

Get started

Create an instance in SurrealDB Cloud and select South America (Brazil) as the region. Everything else - tiers, instance types, versions, backups - works exactly as it does elsewhere, so existing automation needs nothing more than the new region value.

If you are running SurrealDB in production and want high availability alongside in-region deployment, the Scale tier gives you multi-node, multi-availability-zone clusters. The Cloud documentation covers the full set of options.

Deploy in São Paulo.

How Cobrainer built graph-based agent memory on one engine

Mark Gyles — Mon, 07 Sep 2026 09:32:35 +0000

Author: Ignacio Paz

An AI agent is only as useful as what it can remember - and how well it can connect the things it remembers. Most teams hand their agent a memory by reaching for a vector store: embed everything, retrieve by similarity, hope the relevant context comes back. It works, until you notice the agent keeps surfacing things that are near the question but not actually connected to it.

Cobrainer, a skills-intelligence company based in Munich, took a different route. They gave their AI agent a memory that lives in the database as a graph, where the agent builds the relationships between nodes as it goes. They did it without adding a graph database, a vector engine, or a search engine to their stack. It all runs on SurrealDB, alongside a Rust-native agentic graph RAG built on the same store.

Here's how, and why a single engine made the difference.

The problem with flat memory

Cobrainer runs a skills-intelligence platform - the kind of system that reasons about how people, roles, skills, and capabilities relate to one another. That's an inherently graph-shaped problem. But their first retrieval setup wasn't graph-shaped at all.

It pulled context through flat vector retrieval over an S3-and-OpenSearch pipeline, which carried two recurring costs:

Accuracy. Flat vector matches returned context that was loosely related - semantically near, but not necessarily connected in any meaningful way. The team wanted the agent to follow real relationships between entities, so its answers were grounded rather than approximate.

Tokens. Broad vector matches meant stuffing a lot of marginally relevant context into every prompt - expensive, and more so with every call. The team wanted to fetch only the context that mattered.

The obvious fix - adding a graph database on top of the vector and search systems they already ran - would have meant more infrastructure to operate. For a startup moving fast, that fragmentation was the thing to avoid, not embrace.

What they wanted instead

Cobrainer's requirements were less about any single feature and more about not paying a tax for combining them.

• An agent and a graph RAG on one store: an AI agent whose memory lives in the database as a graph, plus an agentic graph RAG. Both are backed by a single engine for graph, vector, and full-text.

• A self-building memory graph: rather than hand-modelling every relationship up front, the agent should make use of the relations between nodes that it builds automatically as it works.

• Grounding through structure: retrieval that traverses real relationships, so answers are anchored in how things actually connect.

• Token efficiency by design: pull the relevant subgraph, not a broad similarity sweep.

• Pace: stand up new storage patterns quickly, without long migrations slowing the team down.

Why did Cobrainer choose SurrealDB?

SurrealDB gave Cobrainer graph, vector, and full-text in a single engine, all queried through SurrealQL - so the agentic use case didn't need pgvector or OpenSearch sitting alongside their database. (Their existing RDS Postgres stayed exactly where it was, doing the job it already did well.)

A few things made the build land the way they wanted:

One engine, queried one way. Documents, graphs, vectors, and full-text all live in one store - schemaless and schemafull side by side - and all reachable through SurrealQL. The agentic case added no new systems to operate.

Graph-based agent memory that builds itself. SurrealDB stores the agent's memory and its session checkpoints. As the agent works, it automatically creates the relations between the nodes it writes - so the memory graph forms on its own, and the agent can later traverse those connections instead of re-deriving them. The team described a multi-layered, graph-based memory model, and the thing that surprised them was how fast it came together.

A Rust-native agentic graph RAG. Cobrainer built the graph RAG natively in Rust on SurrealDB's Rust SDK, combining graph traversal and vector similarity in a single query. That's the piece that replaced the old S3-and-OpenSearch RAG - letting the agent walk relationships and rank by similarity in one round trip instead of orchestrating across services.

Managed, and inside the EU boundary. Because this is HR data, residency isn't optional. SurrealDB Cloud let a lean team run the engine as a managed service with EU-region data residency, keeping everything inside the boundary they're required to maintain.

The results

Cobrainer went from evaluation to a customer-facing production deployment in roughly three months. As of August 2026, that deployment is live. Measured against their previous S3-and-OpenSearch pipeline, the team reports three changes.

• Better accuracy: grounding the agent in real graph relationships, rather than loose vector matches, made its responses more reliable.

• Lower token cost per call: fetching only graph-relevant context - instead of a broad vector sweep - trimmed the tokens going into each prompt.

• A flexible platform for what's next: graph-based agent memory and a Rust-native agentic graph RAG, both on one engine, ready to absorb new use cases without another round of stack expansion.

"As a startup, we didn't want to fragment our stack every time we add a capability. For our agentic graph RAG, the alternative was weighing Postgres with pgvector and search extensions against bolting on OpenSearch - instead we got graph, vector, and full-text from one engine. What surprised us was how quickly we stood up a multi-layered, graph-based memory model for our AI agent, with SurrealDB handling its memory and session checkpoints. And SurrealDB Cloud kept all of it inside our EU data boundary."

Daniel Alker, Senior Cloud Platform Engineer, Cobrainer

The takeaway

Giving an agent a memory doesn't have to mean giving it another database. Cobrainer's story is a small proof of a larger idea. When one engine handles graph, vector, and full-text together, and the agent can build its own memory graph on top of it, the "right" architecture stops being a collection of specialised services. It starts being a single query.

Start with SurrealDB Cloud or explore the docs.

Graph engineering is missing a graph

Mark Gyles — Thu, 13 Aug 2026 13:08:08 +0000

Author: Tobie Morgan Hitchcock

Graph engineering is having a moment.

The definition going round is a good one, and I am not going to try to redefine it. Complex agents increasingly look like graphs: nodes do the work, edges decide what happens next, state moves between them. That is real, it matches what production actually looks like, and the people saying it are right.

Almost nobody makes the next observation. Those agents are still fed context from flat vector chunks, a separate document store, a relational database and some ephemeral session memory. While the execution architecture became a graph, the data architecture underneath it did not.

Graph engineering is not just about graphing the agents. You also have to graph the context they operate on.

Or, less politely: you cannot graph-engineer the agent and leave its memory as a pile of chunks.

This is my attempt to say what graph engineering has to mean if it is going to outlive the meme.

The rename cycle

We have been renaming the same struggle for a few years. Prompt engineering optimised instructions. Context engineering optimised what goes into the window. Harness engineering, loop engineering, and now graph engineering optimise how agents work together.

I do not mind the labels. They appear when builders hit a wall. This wall is real. One agent loop in one chat window stops scaling the moment the work looks like a team: parallel reviewers, a fixer, a security check with veto power, a human gate before anything irreversible. That structure is a graph. It always was.

Every one of those renames moved up the stack, and every one of them left the data layer where it was.

Two graphs. Say which one you mean.

When someone says graph engineering, they could mean one of three things and rarely say which:

A graph of loops, which is really a claim about control flow: what you get when one loop stops being enough.
An execution graph.
A context graph.

The loop graph folds into the execution graph, so two are left standing, and most threads blur even those. Sometimes people mean both. They almost never say so.

The execution graph

What runs next, in what order, with what rights. It determines what your agents do.

Nodes are work: a classifier, a tool call, a coding agent, a reviewer. Edges are transitions: success, fail, escalate, fan out. State is the scratchpad the system mutates. Verifiers sit on a separate path when the stakes are high. Stop rules exist so the thing does not burn tokens until the card declines.

This is a state machine with probabilistic nodes inside it. LangGraph made the case earliest and loudest, and Google's ADK, Mastra, CrewAI, Microsoft's agent frameworks, or a boring job runner you wrote yourself all live on the same plane. Useful, necessary for a lot of production agents, and on its own not sufficient.

The context graph

What is true, what is related, what was true when, and who said so. It determines what your agents know.

Here the nodes are entities, documents, events, users and assets, and the edges between them carry properties of their own: since, confidence, source. Provenance, time, permissions and hybrid retrieval all matter, because the world is not only triples and it is not only embeddings.

Classic knowledge graphs and GraphRAG both sit on this plane. It gets a paragraph in most threads and a whiteboard in almost none.

So:

The execution graph engineers the graph of work. The context graph engineers the graph of knowledge.

Most teams have engineered the first and are still running the second as a pile of chunks in a vector store.

Where they meet: the turn

Every serious agent turn is three moves: read, think, write. Read means traverse relations, pull vectors, load history and permissions. Think means the model reasons over a complete packet, not three partial dumps from three systems that disagree with each other. Write means decisions, new entities, and memories land somewhere the next turn can actually see them.

If your execution graph is elegant and your write path is "maybe the vector DB, maybe the graph DB, maybe a JSON blob in object storage," the whiteboard never became system truth. You choreographed forgetting.

That is the definition I use:

Graph engineering is how you design the execution graph of agent work and the context graph of durable knowledge, then keep them consistent across every turn.

The meme covers the execution graph. Production dies on the other two.

Loops, graphs, context. Stack them.

A loop is fine when one goal, one verifier, and one stop condition cover the job.

An execution graph earns its keep when you need parallel specialists, explicit escalations, or policy gates.

Context engineering is what you do on every node either way: put the right facts, tools, and state in front of the model at the right time. Widening the context window does not solve a wrong-context problem, it just raises the price of getting it wrong.

Harnesses and frameworks host all of this. I am not here to sell you an orchestrator. We did not build SurrealDB to replace your control plane.

We built it because the existing data planes under those nodes were a mess.

GraphRAG is one chapter, not the book

People swap the words, but they should not.

GraphRAG is a retrieval pattern: extract structure from a corpus, then use the graph, and often community summaries, to ground generation. It is a good technique, and one node in a larger system can run it just fine. You can also run GraphRAG with almost no multi-agent workflow at all. Microsoft's own documentation makes the honest version of the argument: vector-only retrieval struggles when answering a question requires connecting information across relationships.

If graph engineering is going to mean anything durable, it is how the whole system moves, remembers, and stays correct. Retrieval is necessary, but it is not the architecture.

Retrieval is not three searches and a re-ranker

Almost everyone ships the same pattern. Fire a vector search. Fire a keyword search. Maybe fire a graph query. Take fifty from each, throw the pile at a re-ranker, keep what floats, stuff it in the prompt.

That pipeline has a structural flaw, and it is not latency. Each leg searches blind to the constraints the other legs know about. The vector index does not know this question is about one customer's contracts; it knows only that a question-shaped vector is near some passage-shaped vectors. So it returns fifty candidates and hopes. The keyword leg does the same. The graph leg returns a neighbourhood with no idea which parts are relevant to the sentence that was asked. Then a re-ranker, which can see the text and none of the structure, is asked to reconcile three lists that were each built without the others' information.

You over-fetch because that is the only defence against three blind legs. Over-fetching grows your token bill. Then you pay a second time, because the pile you carried back is mostly padding, and padding in the context window is not free: it costs money on the way in and accuracy on the way out, since a longer prompt gives the model more chances to ground its answer in the wrong sentence.

Run the similarity inside the traversal rather than beside it.

If embeddings are a field on the entity, and full-text indexes sit on the same rows, and relations are edges between those same rows, then one query expresses the whole thing: start from what the question is about, walk the relations that matter, and score by similarity and text relevance as you go, with permissions and time bounds applied in the same pass. Roughly:

SELECT
    id,
    name,
    ->stated_by->source AS provenance
FROM entity
WHERE ->works_at->company:acme
  AND valid_until = NONE
  AND embedding <|10, 40|> $question

The structure prunes before similarity scores, and similarity ranks within what the structure allows. You are not searching a corpus and hoping the answer's neighbourhood comes with it. You are searching a neighbourhood you already have reason to believe in.

That buys two things.

Fewer tokens. You stop over-fetching, because a constrained candidate set does not need a wide net to be safe. You carry back a small precise set instead of three large ones minus whatever the re-ranker threw away, and you stop paying for the same fact three times because three stores each returned their own copy of it.

Higher accuracy. Every filter that was inexpressible in the vector leg (this entity, this scope, this time window, this permission, currently true rather than superseded) becomes expressible, because it is evaluated in the same statement as the similarity. Constraints you could only apply after ranking are now applied before it.

Our own numbers say the same thing from the other direction. When we were first developing Spectron, our memory layer, we tried the obvious thing: reduce each extracted fact to a short sentence, embed it, and add it as another dense leg in the hybrid mix. It fixed roughly a hundred benchmark questions and broke roughly as many again, landing slightly negative and inside the noise. Meanwhile the two changes that clearly won were both structural: routing from a resolved entity into its typed facts, and expanding a hit into its conversational neighbours. Both worth a couple of points on their own.

The lesson we took: reducing a typed fact to a sentence and matching it by cosine throws away exactly the structure that made extracting it worthwhile. A similarity list also cannot abstain. It always returns its k, so when the right answer is not there it returns something else with total confidence, and something else lands in the prompt. Structure can return nothing, which is frequently the correct answer.

A context graph is not an extraction job

The obvious objection: fine, so I bolt a graph database onto my stack and run an extractor over my corpus.

You can. What you get is a pile of assertions in a vocabulary the model improvised one turn at a time, and the difference shows up months later as retrieval that cannot find things you know are in there.

Here are the numbers from one conversation we ingested end to end. Of the attribute keys the extractor produced, 85% were used exactly once: 236 distinct keys across 325 facts, an average of 1.4 uses per key. Action verbs were nearly as bad, 80% used once. The two most common verbs in the whole set were shared_image and shared, describing the same act, counted separately.

Now consider what that does to a query. WHERE key = 'residence' finds nothing if this turn's extraction called it lives_in, and the next one called it home_city, and a third called it current_residence. The structured path, the one that was supposed to make the graph worth building, stays dark. Retrieval falls through to similarity, and you are back to the pattern in the previous section, paying for a graph you cannot query.

The interesting part is the control case. One relation family in that same measurement was not fragmented: 4.3 uses per label instead of 1.4, only 40% singletons. It was the one family where the extraction prompt already showed the model the vocabulary the system had established, with an instruction to reuse an existing label and only mint a new one when nothing fits. Same model, same text, same run. The only difference was whether the graph told the extractor what it already knew.

A context graph is a loop with itself, not a function over text. Feed the established vocabulary back. Canonicalise predicates at write time so two spellings of one idea land in one bucket. Resolve mentions against entities that already exist, rather than minting a new node for every phrasing, because the same customer becoming five nodes means every traversal lies.

Some hard-won specifics, since this is where the work actually is.

There is no correct base form a priori. We tried folding inflections and the naive rules are landmines: strip a trailing e and care merges with car; undouble a consonant and roll merges with role; strip a trailing s and news becomes new. Any stemmer aggressive enough to bucket reliably produces non-words that then surface in your API responses. What worked was narrower and duller: only ever fold onto a spelling this graph already uses, never toward an invented lemma.

Semantic merging is a judgement about meaning, not a normalisation. home_city and city_of_residence probably mean the same thing. "Use metric units" and "always answer in metric" probably do too. Fold them wrongly and you have discarded something a user explicitly said. We left that undone deliberately, and said so, rather than pretending a string function had settled it.

Time is two axes and they are not the same axis. When a fact was stated is not when the thing happened. If you store one and render it as the other, "when did X happen" gets answered with when it was mentioned, confidently and wrongly. Relative phrases are worse: "last week" is a window, not an instant, and letting the answering model do calendar arithmetic at read time from whatever dates happened to be in its context is exactly the failure you are trying to remove. Resolve at write time, against the source's own date, and store what was actually stated. Store nothing when nothing was stated, so that "no event time" and "the event time equals the ingest time" stop being the same value.

Dates do not belong in the text you embed. We had resolved windows interpolated into the fact sentence that was both the keyword document and the embedded vector. Those date tokens say nothing about what the fact means, and they pollute the retrieval key for every query. Dates live in columns. Renderers read columns. Text and columns cannot disagree if there is only one copy.

Half of what extraction produces is not linked to anything. In our measurement, 52% of action objects were free text rather than a resolved entity: the graph recorded that somebody did something to a string. That string is invisible to traversal. Extraction reported success. The graph got nothing it can walk.

An edge nothing reads is not memory. We shipped an alias edge, wrote a careful writer for it, and then pulled it, because no retrieval path traversed it. It was structure with no consequence.

All of this decides whether the context graph means anything. Meaning cannot be extrapolated from a graph that was never normalised, because the meaning was discarded before the write. And none of it is a one-time import: entity resolution and temporal invalidation are standing jobs, running every turn, or "what is true" decays into "what was true in March."

What actually breaks

Skip the forty-agent overnight graph. That is how you max a bill and learn nothing.

Here is what fails.

State lives only in the transcript. You cannot debug it, audit it, or resume it cleanly if that is the only place it lives. IDs, permissions, and intermediate facts need a home outside the prompt.

Everything is agentic. Money movement, PII, deletes, external side effects: those edges should be code, not model judgement. Give the model freedom where exploration actually pays off, and encode the paths you already know are correct.

Verifiers share the same spoiled context. Agents on the same model, reading the same flawed packet, agree with each other at industrial scale, which is why verifier context needs to be separated and drawn from evidence outside the graph: tests that ran, money that moved, a human weighing in before the expensive push. Organised nonsense is still nonsense.

You drew a DAG and production needed cycles. Retries, clarification, a human pausing and resuming the flow: these are not a design smell, they are how work actually works.

Your context graph is a pile of triples with no ontology. Model the domain before you extract at scale. Schema is what stops you from lying to yourself about what the graph actually means, not overhead for its own sake.

Extraction is stateless. Every turn invents its own vocabulary because nothing showed it the vocabulary that already exists. This is the single cheapest thing to fix and the one most people have not fixed.

The graph is built once and never maintained. Extraction is the easy day. Entities need resolving, or the same customer becomes five nodes and every traversal lies. Facts need expiry, or superseded beliefs sit next to current ones and both look true.

Retrieval is three searches and a re-ranker. Each leg is blind to the others' constraints, so you over-fetch to compensate, pay for the same fact three times, and hand a re-ranker the job of guessing what the query planner should have known.

Write-back is optional. The agent learns a preference, a failed approach, a new link between entities. If that write is not durable and visible under the same permissions as the read, the next node is guessing again.

No stop rule. Parallel agents without token, time, and tool budgets will happily burn through a budget without producing anything you can point to.

The shape everyone ships

Most "graph engineered" demos still look like this:

Four taxes, every time.

Context leaks at the seams. Relationships, history, and metadata fragment whenever they cross a boundary.
The same fact enters the prompt three times, and you pay for it three times.
No query can be planned across the whole picture, so every retrieval is a wide guess narrowed after the fact.
Ops multiplies: five configs, five monitors, five failure modes, and the glue quietly becomes the product.

You want the context graph and the durable data to be one thing. A context graph that cannot hold your application records, your documents, your vectors and your temporal state is a copy of your real data that you will spend the next year syncing.

This is why I flinch when graph engineering is reduced to "use a graph database." A graph-only store solves a real slice, but agents need documents, vectors, time, session state, and above all the ability to ask one question that is simultaneously a traversal, a similarity search and a filter. Split those across stores and that question stops being expressible at any price.

What this looks like inside a bank

Most of the graph-engineering conversation right now is about routing and reliability: how do I get five agents to hand work to each other without falling over. That is a startup's version of the problem.

Ask the same question inside a bank, an insurer or a global enterprise and the requirements change shape entirely. Every enterprise conversation I have had this year has landed on some version of the same list.

Shared context, not per-agent context. Twenty agents across four teams need the same view of the same customer. If each one has its own vector store, you do not have twenty agents, you have twenty inconsistent opinions with an org chart.

Permissions that travel with the data. Not permissions checked at the API gateway and then discarded. If retrieval cannot enforce who may see which fact inside the query itself, then every prompt is a potential exfiltration path and every new agent is a new review.

Provenance carried on the facts themselves. Who said this, in which document, on what date, and is that source still authoritative. "The model said so" is not a defensible answer when the fact ends up in a regulatory filing.

Auditable retrieval. Not just what the agent answered, but what it read to get there, and why those rows and not others. A re-ranker's opinion over three fused lists is close to unexplainable. A query with predicates in it explains itself.

Underneath all of it sits temporal truth: what did we believe on the day the decision was made, not what do we believe now. In a regulated firm the record exists to answer that question, which means facts need validity intervals and reads need to resolve as of a point in time rather than as of now. And the graph needs a boundary in both directions, with control over what is allowed into it and over what is allowed out of it into a prompt.

None of that is served by a prettier flowchart, and none of it is served by a vector store with a metadata filter bolted on. It is served by the context graph being a real database: transactional, permissioned, temporal, and auditable. The current discourse is not covering this, and it is what enterprises are buying.

What we set out to build

I am biased. I will say that up front.

We built SurrealDB as a multi-model engine: documents, graphs, vectors, time-series, full-text, relational, and auth in one system with one query language over them. The reason has nothing to do with graphs being fashionable. It is that the useful query is almost never one shape. It is "things related to this, that resemble that, that were true then, that this user may see," and every system that stores those four facts separately can only answer it by approximation and reassembly.

On the context side that means schema as ontology. Edges carry confidence, time, and source as fields rather than afterthoughts. Vectors live on the entities they came from, so similarity is an operator inside a traversal rather than a separate service you join against afterwards. Traversal, similarity, full-text, and filters resolve in one statement, planned together, instead of three round trips and a sync cron you forgot about until it pages you at 2am. Write-back lands transactionally under the same permission model you read from, which matters, but it is the floor rather than the pitch.

Spectron is the memory layer we built on that engine: entity extraction, predicate normalisation, entity resolution, temporal facts, and hybrid retrieval that is one query rather than three. Most of the engineering effort there has gone into what happens between the text arriving and the graph existing, rather than into retrieval algorithms. When the session ends, the context window forgets, but the system should not.

We are also building the comparison, because I would not take my word for it either: the same multi-agent application twice, once on an orchestrator plus a vector store plus a relational database plus a session cache, and once on the same orchestrator with a single context graph underneath. Same agents, same tasks, same model. Measured on context accuracy, tokens consumed, latency, network hops and lines of glue.

The control plane stays yours. We are complementary on purpose. The question I would ask is not whose orchestrator you use, but whether your retrieval can express a traversal and a similarity search in the same breath, and whether anything understood your data before it became a graph.

A checklist I would actually use

Scope

One recurring job with a real success metric, not a general assistant.
Someone owns the ontology, someone owns the workflow, and early on that can be the same person.

Execution graph

Label nodes: code / single LLM call / tool / full agent.
Edges only where work actually flows.
A verifier path with fresh context for high-risk output.
A human gate on irreversible actions.
Explicit stops and budgets.
Cycles where retries and clarification are real.

Before the context graph exists

Entity and relation types written down before bulk extraction.
Extraction shown the vocabulary the graph already uses, with an instruction to reuse it.
Predicates canonicalised at write time, folding only onto spellings the graph already has.
Mentions resolved to existing entities, with the unresolved rate measured rather than assumed.
Event time and mention time stored as separate columns, resolved at write time, absent when nothing was stated.
Nothing in your embedded text that is not part of what the fact means.

Context graph

Provenance on facts and edges.
Temporal validity, and a job that actually retires superseded beliefs.
A retrieval path that constrains before it ranks, explainable in one sentence.
Write-back after every turn for memories, entities, decisions.
Permissions enforced inside retrieval, not at the gateway.
Graph statistics on a dashboard: predicates per fact, unresolved mentions, duplicate entities. If you cannot see fragmentation, you will not fix it.

Reality checks

Outside-the-graph evidence for ship decisions.
Evals that include multi-hop and time questions.
Cost and latency budgets per node type.
Replayable state when something fails.

Grow the graph when the work forces you, not when a meme does.

What I would ignore

Renaming your orchestrator and calling it strategy.
Forty agents overnight.
Treating GraphRAG, context graphs, and execution graphs as the same slide.
Bigger context windows as a substitute for structure.
A better re-ranker as a substitute for a query that could have been planned properly.
Holy wars about anyone's database. The architectural gap is enough, and the market does not need another feud.

Closing

Graph engineering became the new buzzword because one chatty loop is not enough for real work. Good. Draw the execution graph. Be honest about where the model decides and where code decides.

Then do the half most threads skip. Understand the data before you write it: one vocabulary, resolved entities, real event times, facts that expire. Put the vectors on those entities rather than in another system, so retrieval is one query that traverses and ranks together instead of three searches arguing through a re-ranker. Make permissions and provenance properties of the data rather than of the API in front of it.

The label will get renamed again. It always does, and I would not build a roadmap on the word. But the two graphs are not a naming fashion. Every serious agent system has an execution graph and a context graph whether or not anyone drew the second one, and the gap between them is where the accuracy goes, where the token bill comes from, and where the audit fails.

Frameworks draw the graph of work. Something has to hold the graph of knowledge, keep it true, and survive the write-back. That is the problem we set out to solve with SurrealDB, and with Spectron, the memory layer we built on top of it. If your agent topology is getting serious and your context layer is still five separate systems you are hoping stay in sync, that is the seam to fix.

Introducing Scale: SurrealDB Cloud, built for high availability and scale

Mark Gyles — Thu, 02 Jul 2026 15:53:29 +0000

Author: Tobie Morgan Hitchcock

Today we're launching Scale, a new tier of SurrealDB Cloud built for the workloads you can't afford to have go down.

Our first tier, Start, was designed for building and shipping fast. Scale is designed for what happens next: production traffic, uptime commitments, and the kind of resilience your users never notice because there are always available nodes. It's the tier for teams running SurrealDB as the scalable context layer behind real applications and AI agents in production.

What you get with Scale

Scale is about one thing: keeping your database available and consistent under real-world conditions.

Highly-available, fault-tolerant clusters. Scale runs your database as a multi-node cluster designed to survive node and infrastructure failures without dropping writes or losing consistency. A single point of failure is no longer a single point of downtime.

Multiple availability-zone deployment. Your cluster is distributed across multiple availability zones, so even the loss of an entire zone doesn't take your database with it. Traffic keeps flowing while the cluster recovers in the background.

Horizontal scale. As demand grows, Scale grows with it. Add capacity by scaling out across nodes rather than being capped by the size of a single machine. Start with three nodes, and keep adding to scale your application or agent's needs.

See more information about SurrealDB Cloud Scale architecture here.

Built on SurrealDS

Scale is powered by SurrealDS, SurrealDB's distributed storage engine and the foundation that makes all of this possible.

SurrealDS is a new generation distributed storage architecture, rethought from first principles. Instead of coupling storage to compute on a single box or to a proprietary cloud tier, SurrealDS embeds consensus directly in SurrealDB nodes and separates the two layers cleanly. Here's what that architecture gives you.

Architecture overview

Compute and storage separation. Scale compute for QPS and storage for capacity independently, so you provision for the dimension that's actually under pressure.
No single leader. Each availability-zone node writes locally, writes scale horizontally, and transactions commit once a quorum acknowledges them.
Multi-write nodes. Every write node in the cluster can accept and coordinate transactions - there's no bottleneck routing all writes through one leader.
Reduced operational overhead. Consensus is embedded directly in SurrealDB nodes, eliminating external coordination services.
Fewer network dependencies. A single broadcast replaces multi-hop coordination, reducing latency.
Lower latency than leader-based replication. With no single leader and each AZ node writing locally, transactions avoid the extra round trips of traditional leader-based systems.

Future roadmap

Several SurrealDS capabilities are on the near-term roadmap and will land in the Scale tier as they ship:

Object-storage backed. Persist transactional data directly to commodity object storage - storage costs drop and capacity scales far beyond any single disk.
Cross-region replication. Data flows through object storage rather than between nodes, significantly reducing data-transfer costs while keeping regions in sync.
Instant recovery. A crashed node restores from object storage, with recovery time almost instant regardless of dataset size.
Instant branching. Clone a petabyte-scale database in seconds for development, testing, or experimentation.

Get started

If you're running SurrealDB in production, or getting ready to, Scale gives you the availability and fault-tolerance your workloads need, on the storage architecture no one else has.

Explore the tier in SurrealDB Cloud, and dig into the engine underneath at SurrealDS.

SurrealDB is now available on the Nebius AI Cloud Marketplace

Mark Gyles — Thu, 18 Jun 2026 15:33:29 +0000

Author: Tobie Morgan Hitchcock

We're excited to announce that SurrealDB is now officially available on the Nebius AI Cloud Marketplace.

This launch reflects our commitment to meeting developers and AI teams where they build, and deepens our partnership with Nebius to make SurrealDB easier to adopt, deploy, and scale on one of the fastest-growing AI cloud platforms in the world.

Why Nebius?

Nebius is a full-stack AI cloud built for the demands of modern AI workloads. Powered by the latest GPU and high-performance networking infrastructure, with managed platform, serverless AI, and storage built for the entire AI lifecycle, Nebius gives teams the infrastructure to train, tune, and serve models at scale.

As an NVIDIA Reference Platform Cloud Partner with infrastructure across Europe and the US, Nebius pairs bare-metal performance with the operational simplicity and cost efficiency that AI teams need to move from experiment to production. It's a natural home for SurrealDB - giving you a unified data layer right alongside the compute powering your AI and agentic applications.

Why SurrealDB?

SurrealDB combines document, graph, and relational capabilities in one unified database - designed for agent memory, context layers, modern applications, real-time systems, and AI-powered workloads.With flexible schema, SurrealQL, and built-in real-time features, SurrealDB enables you to build semantic and context layers powered by knowledge graphs for agent memory and multi-agentic workflows.

SurrealDB simplifies your technology stack, reduces Total Cost of Ownership and operational complexity, and allows you to ship products and features in days rather than weeks.

Get started

SurrealDB is available now on the Nebius AI Cloud Marketplace. We can't wait to see what you build, join our Discord community and let us know how you get on!

SurrealDB 3.x by the numbers

Mark Gyles — Fri, 29 May 2026 19:20:49 +0000

Author: Tobie Morgan Hitchcock

One engine, multi-workloads, full durability.

You can explore the full results, methodology, and per-database breakdowns at surrealdb.com/benchmarks.

Why we ran these

Database benchmarks are notoriously easy to game, and difficult to get right. Different hardware, different durability settings, different client libraries, a workload that happens to suit one engine's indexing strategy - any of those will tilt the numbers. So we did three things:

Ran every database on the same hardware - an AMD Ryzen Threadripper 9970X (32C/64T), 128 GiB DDR5, NVMe storage, Ubuntu 24.04.
Used the same open-source harness - crud-bench - with each workload translated into each engine's native query language so no database is penalised for an unfamiliar dialect.
Configured every engine for production-grade durability - fsync on, snapshot isolation, no in-memory shortcuts (except where explicitly noted for embedded comparisons).

We also went out of our way to give every database a fair shot. Rather than running each engine on its out-of-the-box defaults, we used optimised configurations across the board - the same kind of tuning a production team would apply before going live. That meant raising connection and worker pool limits to match the 128-client load, sizing buffer pools, page caches, and shared memory to take advantage of the available 128 GiB of RAM, enabling parallel query execution and prepared-statement caching where supported, setting WAL and checkpoint intervals to values recommended by each project's own performance guides, and turning on the indexes and storage engines (InnoDB, WiredTiger, RocksDB-backed stores, etc.) that each database's documentation recommends for OLTP workloads. The goal was to make sure no engine was held back by a conservative default - if a database underperforms here, it isn't because we left it on its laptop-friendly starter config.

Workloads run with 128 clients issuing 48 concurrent queries each, against datasets of a single table with 5 - 15 million rows of mixed-type records (strings, integers, floats, UUIDs, datetimes, booleans, large text fields, geospatial data, and nested objects and arrays).

About last time

We owe a word on durability. The previous round of benchmark results ran with fsync disabled for every engine - leaving each database's writes in the OS page cache rather than flushed to disk. Every database in the comparison ran with the same setting, so nothing was being "fudged" relative to the other engines, but we didn't make the setting explicit, and the headline numbers ended up describing a workload that most production deployments would not likely run.

This round is different. Every database in these benchmarks runs with full disk durability enabled - fsync on, WAL flushed on every commit, no buffered writes hiding behind the page cache. The configuration files for each engine are checked into the crud-bench repository so anyone can audit them. The numbers above are what each engine sustains when every committed transaction is on disk before the client gets an acknowledgement. That's slower than the cache-friendly numbers you'll find in some marketing posts, ours included, but it's the only honest way to compare databases that are going to outlive a power outage.

How far SurrealDB has come

The biggest story is internal. Across three major releases, we've fundamentally rebuilt the query, parser, and storage layers:

Between SurrealDB 2.x and 3.x alone:

31% faster mean CRUD throughput
58% faster batch operations
11,894% faster non-indexed full-table scans
136% faster indexed queries
Tail latency improvements of 27% (CRUD), 32% (batches), 59% (indexed), and 99% (scans)

The scan number is not a typo. The SurrealDB 3.x query planner and storage engine eliminates the per-row decoding overhead that dominated earlier versions, which is why a workload that used to take minutes now completes in seconds.

How we compare to other databases

SurrealDB is a durable, transactional, multi-model database, so the comparisons that matter most are against the primary databases people actually evaluate it against - Postgres for relational, MongoDB for document, Neo4j for graph. Here's how the same workload looks across those three categories, run on the same hardware with each engine on a tuned production-grade configuration.

vs. PostgreSQL (and MySQL)

CRUD throughput (ops/s) and bulk-read metrics:

SurrealDB is faster than Postgres on every write operation - roughly 1.5× faster creates, 1.3× faster updates, and 1.8× faster deletes - while Postgres still edges ahead on raw single-record reads. Against MySQL the gap widens dramatically: SurrealDB is 5 - 7× faster on writes.

Averaged across creates, updates, and deletes, SurrealDB delivers ~1.5× the write throughput of Postgres - the headline number the benchmarks page leads the Relational category with - and beats Postgres by ~1.5× on full-table counts. Postgres' query planner is 30 years old and still ahead on indexed predicate filtering; we're not pretending otherwise, and we're working on closing that gap in 3.1.

vs. MongoDB (and ArangoDB)

CRUD throughput (ops/s) and bulk-read metrics:

This is the closest race. MongoDB still leads on single-record writes, while SurrealDB is ~1.3× faster on reads - and the picture flips on heavier workloads. On predicate filter scans against unindexed tables - the headline figure the benchmarks page uses for the Document category - SurrealDB is roughly 2.7× faster than MongoDB, with consistently lower mean and p99 latency. Against ArangoDB's document engine, SurrealDB is between 100× and 150× faster on writes.

vs. Neo4j (and ArangoDB)

CRUD throughput (ops/s) and bulk-read metrics:

SurrealDB outperforms Neo4j across every CRUD operation - roughly 2 - 3.5× faster on writes and 1.5× faster on reads - while running the same graph traversals through the same engine that handles documents and tables.

The gap on filtered scans is even more dramatic. Across indexed predicate filter queries - the headline metric the benchmarks page uses for the Graph category - SurrealDB is roughly 35× faster than Neo4j. No separate database, no separate query language, no separate operational story.

Reference point: Redis and KeyDB

For raw key-value throughput, we also ran SurrealDB's in-memory engine (with append-only persistence) against Redis and KeyDB:

That's roughly 3× faster than Redis on writes, updates, and deletes, while offering durable, snapshot-isolated transactions and a full query language Redis doesn't have. Redis still wins on large 1,000-record batch operations and edges ahead on single-record reads.

Embedded mode (vs. SQLite)

SurrealDB's embedded engine runs the same SurrealQL on the same disk format as the server.

Against SQLite, SurrealDB is roughly 85× faster on creates, 110× faster on updates, and 75× faster on deletes, with single-record reads in the same ballpark. On predicate filter scans against unindexed tables it's around 6.5× faster than SQLite, and on indexed filter scans, around 3× faster.

The full breakdowns - including p50, p95, and p99 latencies, batch sizes from 100 to 1,000 rows, indexed and non-indexed predicate filters, and full-text search - are on the benchmarks page.

Closing the remaining gaps

The numbers above are an honest snapshot, not a finish line. There are still workloads where Redis, MongoDB, and Postgres beat us - large batch operations vs. Redis, single-record writes vs. Mongo, indexed predicate filtering vs. Postgres - and we know exactly where each gap comes from. Closing those gaps is the central focus of the SurrealDB 3.1 cycle.

What we're actively working on:

Batch path rewrites to bring 100-row and 1,000-row batched ops closer to Redis throughput, including better client-side pipelining and a leaner server-side batch executor.
A smarter query planner with cost-based optimisation, predicate pushdown into the storage engine, and richer index selectivity statistics - the work that gets us to parity with Postgres on indexed filter scans.
Storage layer improvements for the document workload - more compact in-place updates, sharper write amplification, and tighter integration between the key encoding and the document path resolver - which is where Mongo currently has the edge on single-record writes.
Vector and graph traversal optimisations as those workloads land in the benchmark suite, so the multi-model story holds up at the same rigour as CRUD.

The goal isn't "fastest at one thing." It's to be fastest, or competitive, across every workload that matters, while keeping the one property no specialist engine can match: a single query language - SurrealQL - that handles relational, document, graph, key-value, time-series, vector, and full-text search data in the same database, with the same transactional guarantees, on the same disk format - and the same engine, whether you're running it embedded inside an application, on a single server, at the edge close to your users, or distributed across hundreds of nodes for horizontal scale. We don't think you should have to choose between Postgres, Mongo, Neo4j, and Redis - or between a database that runs on a developer laptop and one that runs across a global fleet. We think a single database should run all four shapes of workload at production speed, anywhere it needs to live, and the SurrealDB 3.x numbers above are the strongest evidence yet that it can.

Why this matters for AI agents

There's a reason we keep pushing on this combination of data models, and it's not historical accident. Agent memory is multi-model by nature. A useful AI agent needs structured facts about the world (relational), semi-structured context and tool outputs (document), entity and event relationships (graph), embeddings for semantic recall (vector), keyword and BM25 retrieval over its corpus (full-text search), episodic and temporal context (time-series), and fast session and cache state (key-value) - and it needs all of that in a single transactionally consistent store, because the moment those shapes live in separate databases, you've built a glue-code problem that breaks every time the schema changes or the model is updated.

Agents also need that memory close to where they run. An agent reasoning inside a browser, a phone, an in-vehicle system, or a per-tenant edge worker can't afford a round trip to a central database for every retrieval. The fact that SurrealDB runs as an embedded engine on the same disk format as the distributed server - with the same query language and the same transactional guarantees - is what makes it usable as the memory layer for agents that move fluidly between local, edge, and centralised deployments. Faster CRUD, faster scans, and faster indexed lookups aren't just abstract benchmark wins; they're how many tools an agent can call, how much context it can recall per turn, and how many concurrent agents a single host can support. That's the workload SurrealDB 3.x is built for, and it's the workload the next round of benchmarks - covering vector search, graph traversal, and full-text retrieval - will measure head-on.

What's next

We intend to expand the benchmarks to also cover CockroachDB, TiDB, MongoDB, and Aerospike for distributed comparisons, which we'll publish in a future round. We also plan to expand the workload set to include graph traversals vector search, and full-text search, two areas where the single-engine, multi-model design of SurrealDB shows its biggest advantages.

Until then: the harness is open source, the results are reproducible, and we'd love for you to run them on your own hardware and tell us what you find.

See the full benchmarks.

SurrealDB 3.1: stability, DiskANN, and a new release process

Mark Gyles — Thu, 28 May 2026 08:43:39 +0000

Author: Tobie Morgan Hitchcock

Three months after 3.0 went GA, we're excited to announce that SurrealDB 3.1 is now available. This is the first minor release in the 3.x series. It builds on the foundations we shipped in 3.0 with a focus on stability, a second approximate-nearest-neighbour index in DiskANN, and a substantial round of security hardening. Alongside the release, we're also rolling out a change to how we develop and ship SurrealDB.

The full list of changes is in the 3.1 release notes. Below are the highlights.

What's new in 3.1

DiskANN approximate-nearest neighbour index

SurrealDB now ships DiskANN as a second ANN index type, sitting alongside HNSW. DiskANN trades a different set of memory and recall characteristics and is well suited to larger-than-memory vector workloads, which has been a recurring request from teams building production agent and search systems on SurrealDB.

The introduction of DiskANN also drove an end-to-end overhaul of the ANN warm-lookup path. Both HNSW and DiskANN benefit from this work, with measurable improvements to warm-cache latency on the existing HNSW index for free.

You can pick the index type at definition time with the new DEFINE INDEX ... DISKANN syntax. See the release notes for the full set of options.

GraphQL: aliases, cursor pagination, and multi-model filtering

SurrealDB v3.1 brings a significant upgrade to the GraphQL surface, making it more expressive, more predictable, and easier to integrate with modern GraphQL clients.

The generated schema now follows Apollo conventions by default, with singular fetch, plural list queries, and createX / updateX / deleteX mutations, so you always know what to expect without consulting per-database config. Fields and tables can now carry GRAPHQL_ALIAS and GRAPHQL_DEPRECATED clauses directly in their definitions, letting you decouple your SurrealQL identifiers from the names your API consumers see:

Schema

DEFINE FIELD first_name ON person TYPE string
  GRAPHQL_ALIAS "firstName";

DEFINE FIELD legacy_score ON player TYPE int
  GRAPHQL_DEPRECATED "Use `rank` instead";

Cursor pagination arrives in this release too. Each table now gets a <plural>Connection query returning edges, pageInfo, and a lazily-evaluated totalCount, so you only pay for the count when you ask for it. Offset pagination via limit and start remains fully supported alongside it.

On the multi-model side, GraphQL queries can now reach across SurrealDB's full data model. Full-text search, vector similarity, and time-series aggregation are all directly queryable through the GraphQL API, unlocking AI-native and analytics-heavy workloads without touching SurrealQL.

This release also closes a range of reported issues: stale schema caches after DDL changes, invalid filter identifiers on nested array fields, missing id range and in filters, and count()-based predicates in WHERE clauses, all resolved and covered by integration tests.

Improved in-memory performance

SurrealDB's in-memory backend now uses a datastore based on optimistic lock coupling. This improves performance by allowing readers to acquire lock-free access, only retrying at the end if a modification has occurred in the meantime. This allows readers to proceed without blocking writers, and vice versa.

Stability and correctness

The bulk of the 3.1 work is bug fixes and stabilisation of the 3.0 line. This continues the stream of fixes that shipped in v3.0.1 through v3.0.5, with a substantial number of additional issues addressed across the query engine, indexing, and storage layers. If you upgraded from 2.x to 3.0 and hit any rough edges, there's a strong chance the relevant fix is in 3.1.

Security hardening

3.1 closes a batch of previously reported security vulnerabilities, alongside a set of previously undiscovered issues that we found internally. The undiscovered ones come from an increased investment in LLM-assisted security review, in line with broader industry practice. Mozilla wrote a good summary of this approach recently. The combined result is a notably larger security section in this release than in previous ones.

The full security section is itemised in the release notes.

Enterprise: audit logging and slow-query pipeline

Customers on the Enterprise tier get two new operational tools in 3.1:

A structured audit log capturing authentication events, schema changes, and other actions of interest to compliance and security teams.
A slow-query pipeline that surfaces queries exceeding a configurable threshold, complete with their plan and execution metadata, to help diagnose performance issues in production.

Both have been on Enterprise customer wishlists for a while, and we're pleased to have them shipping in 3.1.

Other quality of life additions

Among the many changes mentioned in the 3.1 release notes, some of the changes that you won't want to miss are:

A built-in MCP server for AI tools and IDEs,Unified OpenTelemetry metrics and logging pipeline,
Unified OpenTelemetry metrics and logging pipeline,W3C trace propagation across HTTP and WebSocket,
W3C trace propagation across HTTP and WebSocket,Full ALTER coverage for every DEFINE statement,
Full ALTER coverage for every DEFINE statement,
And continued executor and index improvements beyond the in-memory engine.

A new release process

3.1 is also the first release we've shipped under a new development workflow, and we want to be transparent about what's changed and why.

For most of SurrealDB's history, we've developed in public on the main surrealdb/surrealdb repository. When a security vulnerability needed addressing, we'd cut a temporary private fork, fix the issue there, cut a release, patch the SurrealDB Cloud fleet, and then make the release public. The intent was good. Development happened where the community could see it, and security fixes were embargoed only as long as they needed to be. In practice, though, this added real overhead to every security response. Cutting a fresh private fork for each issue, keeping it in sync with main, and merging the fix back when going public all took time we'd rather have spent on the fix itself.

Starting with 3.1, we've moved all development to a private repository. The public surrealdb/surrealdb repo remains the source of truth for releases, issues, and the source code that users read. Day-to-day commits now happen privately.

The cadence looks like this:

We develop and tag releases in the private repository.
When a release is ready, we announce it publicly and ship binaries and Docker images on the usual schedule.
Roughly 1 week after each release, we sync the private repository back to the public surrealdb/surrealdb repo.

The week between release and public sync gives us a private window to triage any issues or vulnerabilities reported against the new version before the corresponding commits become visible. We can ship a fix and roll it out across SurrealDB Cloud before the underlying issue is public. This is the same model used by most database and infrastructure projects at comparable scale, and it lets us close security issues as quickly as they're reported.

A few practical notes:

Issues, PRs, and discussions stay on the public repo. Keep filing them on surrealdb/surrealdb. We watch and respond to them daily.
Security reports. Use the GitHub Security Advisory flow on the public repo, or email security@surrealdb.com. Both reach us privately, and are the right route for anything you don't want disclosed publicly.
Community PRs. We're still very keen for community contributions. They get reviewed against the public repo as before, and brought into the private tree as part of the normal cycle.

We don't expect this to be visible to most users in day-to-day use. If you follow commit-by-commit development on the public repo, the experience will change: you'll see batched updates around each release rather than a continuous stream. We think the security benefit is worth that tradeoff, and we plan to keep working this way for the foreseeable future.

Try it

SurrealDB 3.1 is available now. Head to the install page to grab it for your platform.

If you're running on SurrealDB Cloud, instances are being rolled to 3.1 as part of our standard release cycle. No action needed.

Read the full release notes for the complete list of changes, and come say hello on Discord if you hit anything along the way.

Kreuzberg & SurrealDB: from unstructured documents to hybrid retrieval

Mark Gyles — Tue, 12 May 2026 12:35:55 +0000

Author: Ignacio Paz

We’re excited to share a new partner integration: kreuzberg-surrealdb, a connector that bridges the Kreuzberg document intelligence framework directly into SurrealDB. This integration was created by the Kreuzberg team and we are excited to have this functionality available now in SurrealDB.

Kreuzberg extracts, chunks, and generates embeddings from 88+ document formats, while SurrealDB provides a multi-model database for AI applications, combining documents, graphs, vectors, and full-text search in a single system.

Together, they make it easy to build document search and RAG pipelines.

What the integration does

kreuzberg-surrealdb handles the full ingestion workflow:

Automatic schema setup
Content deduplication using SHA-256 hashing
Storage and indexing in SurrealDB
Documents ready for search immediately after ingest

The integration supports two modes:

DocumentConnector:indexes full documents for BM25 keyword search.
DocumentPipeline:chunks documents, generates embeddings, and enables semantic and hybrid search using HNSW vector indexes and Reciprocal Rank Fusion.

Why it matters

Building document search systems often requires combining multiple tools for extraction, chunking, embeddings, and storage.

With kreuzberg-surrealdb, the entire workflow runs through a single integration—no schema boilerplate, no duplicate ingestion, and built-in support for keyword, semantic, and hybrid search.

Get started

See how to get started in SurrealDB Docs: Kreuzberg Integration, and check out our example of How to build a knowledge graph for AI with SurrealDB and Kreuzberg.

Schema migrations in SurrealDB: A local dev workflow

Mark Gyles — Fri, 08 May 2026 12:42:16 +0000

Author: Chiru Boggavarapu

This post walks through a proper migration workflow for local development using SurrealKit, an official tool from the SurrealDB team that handles schema sync, rollouts, seeding, and testing.

If you've spent any time working with SurrealDB, you know that with great flexibility comes the responsibility of managing your schema carefully. You're iterating quickly, adding tables, refining field definitions, and removing what you no longer need, and before long your local database can drift out of sync with what your code expects.

What is SurrealKit?

SurrealKit is a CLI tool that manages your SurrealDB schema through .surql files. You define your schema as files, and SurrealKit keeps your database in sync with them.

It has two main modes:

Sync: a fast, declarative approach for local and dev environments. Your files are the source of truth. Add something and it gets created, change it and it gets updated, remove it and it gets deleted.
Rollout: a more controlled migration path for shared or production databases, with planning, staged execution, and rollback support.

For local dev, you'll mostly live in sync. Rollouts come into play when you're pushing to shared environments.

Getting Started

Cargo is the Rust package manager. It downloads a Rust package's dependencies, compiles the packages, and makes distributable packages.

Install via Cargo

cargo install surrealkit

Or grab a prebuilt binary from the GitHub releases page if you don't want to compile it.

Initialise a project

surrealkit init

This creates a /database directory with the scaffolding you need. SurrealKit connects to your database using environment variables, so add these to your .env:

.env

SURREALDB_HOST=localhost:8000
SURREALDB_NAME=myapp
SURREALDB_NAMESPACE=development
SURREALDB_USER=root
SURREALDB_PASSWORD=secret

It supports both DATABASE_HOST and PUBLIC_DATABASE_HOST variants, which is handy if you're working in a SvelteKit or similar setup where env vars are split by visibility.

The Local Dev Workflow

Writing Your Schema

Schema files live in database/schema/. Each file is a .surql file with your table definitions, indexes, access rules — whatever you'd normally write in SurrealQL.

The structure is entirely up to you. You might have one file per table, or group related things together. Either way, SurrealKit tracks what's in those files and reconciles it with what's actually in your database.

Syncing Changes

When you've made a change and want to apply it:

surrealkit sync

That's it. SurrealKit diffs your schema files against the current database state and applies what's changed. If you deleted a table definition from your files, it removes it from the database too. Everything stays in sync.

For active development sessions, watch mode is where you'll spend most of your time:

surrealkit sync --watch

This watches your database/schema/ directory and resyncs automatically whenever you save a file. It handles deletions too — if you remove a definition, it gets cleaned up. No manual intervention needed.

Moving to Shared Environments

The sync approach is great for local databases you own completely. But the moment you're working against a shared or staging database, you need more control. That's what rollouts are for.

Baseline First

Before your first rollout on an existing database:

surrealkit rollout baseline

This records the current state so SurrealKit knows what it's working from.

Plan → Start → Complete

When you're ready to ship a schema change:

surrealkit rollout plan --name add_user_indexes

This generates a TOML manifest in database/rollouts/ describing exactly what will change. You can review it, commit it, get it reviewed. Treat it like a pull request for your schema.

When you're ready to apply, you'll reference the manifest by its full filename. SurrealKit names these with a timestamp prefix: 20260302153045 translates to 2nd March 2026 at 15:30:45. This is the time the plan was generated, and it's there so rollouts always have a guaranteed sort order. If two people generate a plan on the same day, the timestamps keep them distinct and sequential. You'll see the full name in your database/rollouts/ directory after running plan.

When you're ready to apply:

surrealkit rollout start 20260302153045__add_user_indexes

This runs the non-destructive part of the migration. Once your application is deployed and you're confident everything is working:

surrealkit rollout complete 20260302153045__add_user_indexes

This finishes the migration, including any cleanup of legacy objects.

If something goes wrong mid-rollout:

surrealkit rollout rollback 20260302153045__add_user_indexes

You can also validate a manifest without touching the database:

surrealkit rollout lint 20260302153045__add_user_indexes

And check what state a rollout is in:

surrealkit rollout status

SurrealKit tracks rollout state in the database itself, so it's always resumable.

Seeding

SurrealKit has a built-in seeding system for populating your local database with test data:

surrealkit seed

Seed files live alongside your schema in the /database directory. Useful for onboarding new team members quickly — they clone the repo, init SurrealKit, run sync and seed, and they're good to go.

Testing

One of the more interesting features is the declarative testing framework. You write TOML test suites in database/tests/suites/ and run them with:

surrealkit test

Each suite runs in an isolated ephemeral namespace, so tests can't interfere with each other or your actual data. You can test SQL assertions, permission rules, schema metadata, and even HTTP API endpoints.

A simple example, checking that a guest can't create an order:

name = "security_smoke"

[[cases]]
name = "guest_cannot_create_order"
kind = "sql_expect"
actor = "guest"
sql = "CREATE order CONTENT { total: 10 };"
allow = false
error_contains = "permission"

You can also test permission matrices across a whole table at once, which is handy for verifying your access rules haven't regressed after a schema change.

For CI, add --json-out to get machine-readable output:

surrealkit test --json-out database/tests/report.json

The command exits non-zero on any failure, so it integrates cleanly with GitHub Actions and other CI/CD systems seamlessly.

Putting It Together

Here's what a typical day looks like with this setup:

Pull the latest changes
Run surrealkit sync to update your local DB
Edit schema files in database/schema/
Use surrealkit sync --watch while you work
When you're done, run surrealkit test to make sure nothing broke
Commit your schema files alongside your code

For production changes, swap step 4 for surrealkit rollout plan and follow the plan → start → complete flow.

Would love to hear how you get on with SurrealKit. Jump into Discord and let us know how it goes or if you need a hand along the way.

New SurrealDB docs search using hybrid search and HNSW/BM25 reranking

Mark Gyles — Fri, 01 May 2026 15:32:31 +0000

Author: Dave MacLeod

Trying to produce relevant search results is one of the hardest things to get right when building an app. Searching can be done in a lot of ways: basic text search, full-text search, vector search, and often even graph and geospatial.

This post introduces an example that includes two of these models: full-text and vector search together. These two tend to involve quite a bit of SurrealQL code, because:

To use full-text search, you first have to define how you want text to be split up and modified. Do you want it to be case-sensitive? Split by whitespace, or some other way? Modified to root forms based on the language of the text? (And so on...)
To use vector search, you have to generate embeddings for pieces of text and know how to query them.

What makes this example special is that it is the search function you use every time you type some text into the "Search the docs" part of the SurrealDB documentation! This functionality was recently implemented, and because it's open source we are able to show you the ins and outs of how it was done.

How Doc Search Is Implemented

The README in the /search section of the docs repo goes into a good amount of detail.

In addition, you can see the mechanics of the search itself inside this search.ts file in another section.

The entire implementation is done using SurrealDB, along with Bun to deploy the website and an OpenAI API key. You can go to this part of the page to deploy it locally and test it out.

Here is what the final result looks like:

Since the README file already goes into detail about how to do the deployment yourself, in this post we will instead take a closer look at the SurrealQL statements themselves to see how they work and how they might help you do something similar in your own tools and apps.

The role SurrealDB Plays

As the README mentions, the search functionality uses hybrid search:

Hybrid search for the SurrealDB documentation. Combines BM25 full-text search with OpenAI vector embeddings, fused via Reciprocal Rank Fusion (RRF) inside SurrealDB.

Hybrid search allows you to combine full-text search with vector search, giving you the best of both worlds.

Vector search is useful because it is capable of going beyond the outward appearance of words to get to their semantic meaning. For example, the term lead in the following paragraph is used to mean two different things. We can recognize that as humans, and LLM models are also able to do the same, returning the output as an embedding (a large array of numbers representing semantic space).

The project lead frowned and took a hard look at the results. They were clear as day. There was far too much lead content in the water. That's why the town had gotten sick!

However, full-text search is useful too because it excels at slicing and modifying text into tokens. To show what that looks like, here's what the full-text analyzer looks like that is used in the docs search.

DEFINE ANALYZER OVERWRITE simple
    TOKENIZERS blank, class, camel, punct
    FILTERS snowball(english);

The easiest way to understand what all the tokenizers and filters do in an analyzer is by putting a string into the search::analyze() function, which returns an array. To avoid showing a huge array in this blog post, we'll call .join(' ') on the output to join them back into a single string.

search::analyze(
    "simple", 
    "The project lead frowned and took a hard look at the results. They were clear as day. There was far too much lead content in the water. That's how the town had gotten sick!")
.join(' ');

As the output shows, using this analyzer will allow you to match on terms like "FroWNinG" which will turn into "frown" before being compared against a search string.

"the project lead frown and took a hard look at the result . they were clear as day . there was far too much lead content in the water . that ' s how the town had gotten sick !"

So if vector search and full-text search each have their advantages, how can you combine the two? It would be nice if you could somehow plug them into a function that does this.

And it turns out that SurrealDB has exactly that! It is called search::rrf() and was added during the 3.0 beta period. Here is what it looks like in practice.

-- ── Reciprocal Rank Fusion ──
-- Combines the four ranked lists into a single ranking.
-- RRF scores each result as: sum(1 / (k + rank_in_list))
-- across all lists the result appears in.
--   arg 1: array of ranked lists to fuse
--   arg 2: k=60 (smoothing constant, standard RRF default)
--   arg 3: limit=80 (max candidates to consider)
LET $fused = search::rrf(
    [$page_ft, $page_vs, $section_ft, $section_vs], 
    60, 
    80
);

As you can probably guess, it is fusing the results of previous queries for full-text search on pages and sections ($page_ft, $section_ft) together with vector searches on pages and sections ($page_vs, $section_vs).

Full-text searches are done using the @@ operator (the "matches" operator). This operator works as long as you have a index on a field that uses a FULLTEXT ANALYZER:

DEFINE INDEX OVERWRITE page_ft_title       ON page FIELDS title       FULLTEXT ANALYZER simple BM25;
DEFINE INDEX OVERWRITE page_ft_breadcrumb  ON page FIELDS breadcrumb  FULLTEXT ANALYZER simple BM25;
DEFINE INDEX OVERWRITE page_ft_description ON page FIELDS description FULLTEXT ANALYZER simple BM25;
DEFINE INDEX OVERWRITE page_ft_content     ON page FIELDS content     FULLTEXT ANALYZER simple BM25;
DEFINE INDEX OVERWRITE page_ft_path        ON page FIELDS path        FULLTEXT ANALYZER simple BM25;

In between these two @ operators you can insert a number to access the search::score() for each field. This will make sense once you take a look at these two queries. Each of them is querying multiple fields, each of which will return a score thanks to BM25 defined in the field. To know which field is which, you put a reference number in between the @ operators like @0@ and @2@. They can then be added together or multiplied or whatever logic you would like to use.

LET $page_ft =
        SELECT
            id,
            "page" AS kind,
            path AS url,
            path AS page_path,
            collection,
            title,
            breadcrumb,
            description,
            content,
            (
                (search::score(0) * 15)
                + (search::score(1) * 25)
                + (search::score(2) * 10)
                + (search::score(3) * 8)
                + (search::score(4) * 3)
            ) AS ft_score
        FROM page
        WHERE
            path @0@ $query
            OR title @1@ $query
            OR breadcrumb @2@ $query
            OR description @3@ $query
            OR content @4@ $query
        ORDER BY ft_score DESC
        LIMIT 30;

LET $section_ft = 
        SELECT
            id,
            "section" AS kind,
            string::concat(page.path, "#", anchor) AS url,
            page.path AS page_path,
            page.collection AS collection,
            title,
            breadcrumb,
            content,
            (
                (search::score(0) * 25)
                + (search::score(1) * 10)
                + (search::score(2) * 3)
            ) AS ft_score
        FROM section
        WHERE
            title @0@ $query
            OR breadcrumb @1@ $query
            OR content @2@ $query
        ORDER BY ft_score DESC
        LIMIT 30;

The vector search query also uses its own index. Note that you don't specifically need an index to use vector search, as you can go with a brute force function such as vector::similarity::cosine() instead. However, an HNSW index instead can speed up the process dramatically if you don't mind sacrificing a bit of precision. Here the index is defined with a dimension of 1536 to match the length of the array returned by OpenAI's text-embedding-3-small model.

-- HNSW index for approximate nearest-neighbour vector search.
-- Dimension 1536 matches OpenAI text-embedding-3-small output.
-- Cosine distance is the standard metric for normalised text embeddings.
DEFINE INDEX OVERWRITE page_embedding_hnsw ON page FIELDS embedding
    HNSW DIMENSION 1536 DIST COSINE;

Having an HNSW index also lets you use the <||> operator (the KNN, or K-nearest neighbour) operator when performing the query. The comments above the code explain what each of the two numbers mean. Here, the |30,100| syntax means to return 30 neighbours among up to 100 candidates.

-- ── Page vector search ──
-- Finds the 30 pages whose embeddings are closest to the
-- query embedding. The <|30,100|> syntax means: return 30
-- neighbours, exploring up to 100 candidates in the HNSW
-- graph (higher = more accurate but slower).
LET $page_vs = 
    SELECT
        id,
        "page" AS kind,
        path AS url,
        path AS page_path,
        collection,
        title,
        breadcrumb,
        description,
        content,
        vector::distance::knn() AS distance
    FROM page
    WHERE embedding <|30,100|> $qvec
    ORDER BY distance ASC
    LIMIT 30;

-- ── Section vector search ──
-- Same as page vector search but for H2 sections.
-- Pulls the parent page's path and collection via the
-- record link (page.path, page.collection).
LET $section_vs =
    SELECT
        id,
        "section" AS kind,
        string::concat(page.path, "#", anchor) AS url,
        page.path AS page_path,
        page.collection AS collection,
        title,
        breadcrumb,
        content,
        vector::distance::knn() AS distance
    FROM section
    WHERE embedding <|30,100|> $qvec
    ORDER BY distance ASC
    LIMIT 30;

So that is how the magic happens. Well, part of it. Be sure to check out the rest of the code which shows what happens after the SurrealDB query returns. For example, as the database is unaware of which collections it is searching through, some modifications are made on the SDK side to give SDK docs less precedence compared to more core docs. This is something that you could perhaps do on the database side if you wanted (starting with DEFINE TABLE sdk_page instead of DEFINE TABLE page for example) but that will always depend on your own case.

// ──────────────────────────────────────────────────────────
// Post-retrieval boosting
//
// After RRF fusion, we apply multiplicative boosts to adjust
// rankings based on signals the database query can't capture:
// title similarity, content type (page vs section), source
// collection, and comparison-query detection.
// ──────────────────────────────────────────────────────────

/**
 * Non-SDK doc collections get a small ranking boost because
 * generic queries like "authentication" should prefer the
 * core concept page over an SDK API reference page that
 * happens to mention auth as one of many methods.
 */
const CORE_COLLECTIONS = new Set([
    "doc-surrealdb",
    "doc-surrealql",
    "doc-tutorials",
    "doc-cloud",
    "doc-surrealist",
    "doc-surrealml",
    "doc-surrealkv",
    "doc-integrations",
]);

Simplifying the Example

Since you might not have the head space at the moment to play around with the existing code demonstrated in this blog post, let's finish up with the same pattern as above in a very simplified form. This will let you copy and paste the example into the online Surrealist UI to give it a try yourself.

The content below holds all the same features: a full-text analyzer, a full-text and vector index, a full-text and vector query, and a final query that combines the two into hybrid search results. However, it has all been simplified to a single field for text content and embeddings, and a single document table that is being searched on. In addition, the embeddings have been cut down to an array of just ten numbers in length. An array that small is very imprecise, but just long enough to serve for our simple example.

-- Define an analyzer
DEFINE ANALYZER OVERWRITE simple TOKENIZERS blank, class, camel, punct FILTERS SNOWBALL(en);

-- Attach it to an index
DEFINE INDEX OVERWRITE ft ON document FIELDS content FULLTEXT ANALYZER simple BM25(1.2, 0.75);

-- Create a vector index too
DEFINE INDEX OVERWRITE page_embedding_hnsw ON document FIELDS embedding HNSW DIMENSION 10 DIST COSINE;

-- Add some sample data
{
    CREATE document:one SET content =   "Elves and halflings and witches and such", embedding = [-0.0019, -0.0142, 0.0080, -0.0664, 0.0173, 0.0109, 0.0066, 0.0045, 0.0204, 0.0087];
    CREATE document:two SET content =   "Is that a wizard?",                        embedding = [0.0010, -0.0051, 0.0207, -0.0787, -0.0061, 0.0127, -0.0172, -0.0097, 0.0077, -0.0120];
    CREATE document:three SET content = "Databases are used to store information",  embedding = [-0.0317, -0.0112, 0.0118, -0.0348, 0.0061, 0.0114, 0.0310, 0.0117, 0.0034, 0.0195];
    NONE
};

-- A sample query and its embedding
LET $witch_text = "witches";
LET $witch_embed = [-0.0200, -0.0059, -0.0081, -0.0475, 0.0020, 0.0295, -0.0183, 0.0170, 0.0048, 0.0286];

-- Get the full-text score
LET $fts_score =
        SELECT
            id,
            content,
            search::score(0) AS ft_score
        FROM document
        WHERE
            content @0@ $witch_text;

-- Get the vector score
LET $vector_score = 
    SELECT
        id,
        content,
        vector::distance::knn() AS distance
    FROM document
    WHERE embedding <|30,100|> $witch_embed
    ORDER BY distance ASC;

-- Combine the results as a hybrid score
search::rrf([$fts_score, $vector_score], 60, 80);

We can see in the output that while only one document had a match on the word "witches" (and thus a value for ft_score), the vector search portion has been able to conclude that the sentence about wizards is semantically a bit closer than the one about databases. This would be a lot more accurate of course if the embedding were longer than 10 elements, but even at this length we can still see its potential.

[
    {
        "content": "Elves and halflings and witches and such",
        "distance": 0.2789615948854415,
        "ft_score": 0.4782197177410126,
        "id": "document:one",
        "rrf_score": 0.024691358024691357
    },
    {
        "content": "Is that a wizard?",
        "distance": 0.385041442417419,
        "id": "document:two",
        "rrf_score": 0.012195121951219513
    },
    {
        "content": "Databases are used to store information",
        "distance": 0.3876749269204385,
        "id": "document:three",
        "rrf_score": 0.012048192771084338
    }
]

Hopefully the simplified example has made it easy to follow the logic we use in our own doc search and has given you some ideas for how to implement patterns yourself. Have any questions or comments about hybrid search? Feel free to Create a free SurrealDB Cloud instance today to get started, and drop by our Discord community to discuss anything on your mind with the SurrealDB community and staff.

How to get near-perfect, deterministic accuracy from your AI agents

Mark Gyles — Thu, 23 Apr 2026 22:23:21 +0000

Author: Matthew Penaroza

I have spent a lot of time working on large-scale agent architectures with some of the largest organizations in the world, and the single most common mistake I see teams make is assuming their accuracy problems are model problems. They almost never are. Your agent is typically not struggling because the LLM is weak. It is struggling because your retrieval layer is feeding it bad context. Fix your retrieval, and accuracy jumps. Make your retrieval learn from its own results, and accuracy becomes deterministic. Here is what that looks like in practice.

TL;DR - Agent accuracy problems are almost always retrieval problems, not model problems. At scale, vector search alone is not enough. You need structured filters, graph traversal, temporal constraints, and vector similarity composed in a single atomic query to get to ~90–95% accuracy. To close the remaining gap to ~99%+, you need reasoning graphs and retrieval graphs: structured graph edges that capture the agent's logic and retrieval patterns, feeding back into the pipeline so the system learns which approaches produce correct outcomes. Both require a multi-model database with ACID transactions across every data model, which is exactly what we designed SurrealDB to be.

The problem: why agent accuracy hits a ceiling

Every agent follows the same loop: retrieve context, reason over it, act. The quality of the action is bounded by the quality of the context. I have seen perfectly capable models produce terrible outputs because the context they were given was noisy, and I have seen mid-tier models produce excellent outputs because the context was precise. The model matters less than most teams think.

At small scale, none of this is a problem. If you have a few thousand documents, vector search returns good results, everything fits in a single database, and accuracy looks great. The problems emerge when data grows into the millions or billions of records. Suddenly more candidates look semantically similar, top-k selection becomes unstable, unrelated chunks bleed into the context window, and your agent starts hallucinating. The RAG survey literature (Gao et al., 2023) documents these failure modes extensively. Vector similarity is a useful ranking signal. It is not a control plane.

What makes it worse is the typical production stack. Most teams are running Postgres for state, Neo4j for relationships, Pinecone for vectors, MongoDB for documents, sometimes Elasticsearch and a memory layer on top. Five or six independent systems, each with its own consistency model. When the graph database is 200ms behind the relational store, or the vector index has not ingested a document that was just written, your agent is reasoning over a version of reality that does not exist. It does not know the data is stale. It just makes a worse decision. And in multi-step workflows, each bad retrieval compounds into the next.

So the question worth asking is not "which model should we use?" It is: how do you guarantee that the context reaching the model is valid, complete, current, and precise, even at scale? In my experience, the answer has two layers. The first gets you to roughly 90–95%. The second closes the gap to 99%+.

The accuracy staircase

Each layer compounds on the last. Skip any step and accuracy hits a ceiling.

Getting to ~90–95%: scope-first retrieval

The idea is straightforward: instead of running similarity search against everything and cleaning up the results, you constrain the candidate set before ranking. Define what counts as a valid result first. Rank inside that scope second.

Think about what a single agent retrieval step actually needs, and what goes wrong without each layer.

Structured filters narrow by type, permissions, and tenancy. Without them, I have watched a support agent retrieve internal engineering postmortems and serve them verbatim to a customer, because the postmortem was semantically similar to the customer's question. The vector search was technically correct. The result was a data leak.

Temporal constraints ensure the data is current. Without them, an agent recommending products will happily surface items that were discontinued months ago, because the old product description is still the closest embedding match. The customer clicks through to a dead link. Trust erodes.

Graph traversal follows real relationships: this customer owns this product, which has this known issue, which maps to this knowledge base article. Without it, your agent retrieves articles about similar-sounding problems on entirely different products. The answer reads plausibly but is wrong in the specifics, which is the worst kind of hallucination because it is hard to catch.

Only after all of that scoping should vector similarity and full-text search run, ranking the remaining candidates by relevance. By this point, the candidate pool is already valid. Similarity is just picking the best among the valid.

The problem is you cannot compose these operations across a fragmented stack. In a multi-database architecture, each is an independent query against an independent system. The application layer stitches the results together and hopes the data was consistent across all of them. It never fully is. You need a database that handles all of this natively, in a single query, against a single consistent snapshot. That is why we built SurrealDB the way we did: structured filters, temporal constraints, graph traversal, vector similarity, full-text search, and business rule enforcement all compose in one SurrealQL transaction.

Here is what that looks like. One query, every scoping layer, against one consistent snapshot:

SELECT
    id,
    title,
    vector::distance::knn() AS vec_dist,
    search::score(1) AS ft_score,
    (1 - vector::distance::knn()) * 0.65 + search::score(1) * 0.35 AS blend_score,
    search::highlight('<em>', '</em>', 1) AS content_snippet
FROM knowledge_base
WHERE type = 'support'
    AND tenant = $tenant
    AND $agent_principal INSIDE allowed_principals
    AND updated_at > time::now() - 30d
    AND id IN $customer->owns->product->has_issue->knowledge_base
    AND content_embedding <|100,40|> $query_embedding
    AND content @1@ $query_text
ORDER BY blend_score DESC
LIMIT 10;

That single statement does what would take four or five round-trips across a fragmented stack: permission checks, temporal filtering, graph traversal through the customer's actual product relationships, and hybrid vector + full-text ranking. One query. One transaction. One consistent snapshot. The line that does the most work is the graph traversal: $customer->owns->product->has_issue->knowledge_base. That is not a join. It is a native graph traversal across edges, and it means the agent only sees knowledge base articles connected to the specific products this customer actually owns. Everything else is excluded before similarity even runs.

Two reasons fragmented stacks kill accuracy

Microsoft's GraphRAG work (Edge et al., 2024) demonstrated the same principle at the retrieval level: combining graph structure with similarity search significantly outperforms vector-only approaches. The CoALA framework (Sumers et al., 2024) describes it architecturally: agent memory must be consistent and accessible within a single reasoning cycle. A multi-model database that handles all data models under one transaction boundary is the simplest way I know to actually do both.

This gets you to roughly 90–95% accuracy on large corpora. The consistency gaps are gone. The retrieval noise is way down. For many use cases, this is enough.

But for agents that need to operate autonomously, make consequential decisions, or run in production without human oversight, the last 5–10% is where the difference between "mostly works" and "deterministic" lives. Closing that gap requires a different kind of mechanism.

Getting to ~99%+: reasoning graphs and retrieval graphs

Scope-first retrieval fixes the shape of the funnel. Reasoning and retrieval graphs fix the funnel itself. They are the mechanism that turns a static retrieval pipeline into one that learns from its own results, and in my experience they are the single biggest differentiator between agents that are "pretty good" and agents that are genuinely deterministic.

Every time an agent runs a retrieval pipeline and makes a decision, two graph structures get written in the same transaction as the action: a reasoning graph (the agent's decision and the evaluated edges for each piece of evidence it considered) and a retrieval graph (the operational details of how context was retrieved).

I want to be specific about what these are. They are not log entries. They are not analytics events. They are graph edges stored in the same database as the data they describe, connecting the agent, the action, the customer, and the evidence records the agent evaluated. Written atomically with the agent's action and fully traversable with the same query language you use for everything else. ACID-guaranteed edges in the same graph your agent operates on. This is exactly why SurrealDB is a graph-native multi-model database: the reasoning and retrieval graphs have to be part of the same graph as the documents, the relationships, the vector embeddings, and the agent state, all under one transaction boundary. If they are not, the causal link between "what was retrieved," "how the agent reasoned about it," and "what it decided" breaks.

The reasoning graph is a set of edges radiating from the agent. A decided edge connects the agent to the action, carrying confidence and the eventual outcome. But the reasoning itself is not a field on that edge. Each step of the agent's logic is its own evaluated edge connecting the agent directly to the evidence it considered: agent->evaluated->kb:4821 with a verdict (used or rejected), the reason, and how much it moved the agent's confidence. The reasoning is not a string or an array. It is a set of traversable edges connecting the agent to every piece of evidence it touched. The retrieval graph is a retrieved_via edge from the action back to the customer, carrying which scope filters were applied, the graph path traversed, the candidate count before and after scoping, whether the top result was relevant, and the retrieval latency.

Anatomy of the Reasoning and Retrieval Graphs

Every reasoning step and every retrieval detail is a graph edge. Traversable, not buried in strings.

The reason the reasoning has to be a graph, not a field, is that the pipeline planner needs to traverse individual evaluation steps across thousands of runs. If kb:2201 gets rejected in 94% of firmware queries, the planner can only discover that by traversing evaluated edges, not by parsing strings in a log. In SurrealQL, this is what the actual transaction looks like. The action is a node. The reasoning and retrieval are edges, created with RELATE, connecting the agent to each piece of evidence and the action back to the customer:

BEGIN;

CREATE support_action:escalate_7291 CONTENT {
  action: 'escalate_to_engineering',
  customer: customer:acme,
  ticket: ticket:9913,
  created_at: time::now()
};

RELATE agent:support_tier2->decided->support_action:escalate_7291 CONTENT {
  confidence: 0.94,
  created_at: time::now()
};

RELATE agent:support_tier2->evaluated->kb:4821 CONTENT {
  action: support_action:escalate_7291,
  step: 1, verdict: 'used',
  reason: 'Identical symptoms on same SKU',
  confidence_delta: 0.3
};

RELATE agent:support_tier2->evaluated->kb:2201 CONTENT {
  action: support_action:escalate_7291,
  step: 2, verdict: 'rejected',
  reason: 'Wrong product line',
  confidence_delta: 0
};

RELATE agent:support_tier2->evaluated->kb:5540 CONTENT {
  action: support_action:escalate_7291,
  step: 3, verdict: 'rejected',
  reason: 'Issue resolved, no longer active',
  confidence_delta: 0
};

RELATE support_action:escalate_7291->retrieved_via->customer:acme CONTENT {
  scope_filters: { type: 'support', tenant: 'acme', max_age: '30d' },
  graph_path: 'customer:acme->owns->product:x100->has_issue->kb',
  candidates_before: 2847331,
  candidates_after: 14,
  top_result_relevant: true,
  latency_ms: 8.2,
  created_at: time::now()
};

COMMIT;

The decision logic is not buried in strings. Every reasoning step is an edge in the graph. The decided edge connects the agent to the action. Each evaluated edge connects the agent to a specific piece of evidence it considered, carrying the verdict, the reason, and the confidence delta. The retrieved_via edge connects the action back to the customer with the retrieval metadata. The evaluated edges are not magic. Your agent framework already evaluates candidates and produces a decision. The RELATE statements just persist what the agent already computed. The only design requirement is that your agent emits structured evaluations per candidate rather than a single monolithic response.

That graph structure gives you something most teams have never had: you can replay any agent run, traverse the exact reasoning it followed, and identify exactly where and why it went wrong. But the real value is what happens when the reasoning and retrieval graphs feed back into the system. And they feed back in two different ways, serving two different purposes.

The pipeline planner traverses the reasoning and retrieval graphs to decide what context the agent sees. This is structural optimization. The planner can start from kb:2201 and traverse all evaluated edges pointing at it across every run. If it discovers that kb:2201 gets rejected 94% of the time for firmware queries, it deprioritizes or excludes it from the candidate set for that query type. It can walk agent:support_tier2->evaluated->* to see every piece of evidence this agent has ever considered, grouped by action, filtered by verdict. It learns which retrieval patterns, graph paths, and scope filters correlate with correct outcomes and tightens the funnel accordingly.

The LLM itself receives relevant past reasoning edges as context to guide how it thinks about the candidates it receives. This is behavioral optimization. When the agent retrieves context for a firmware issue on product:x100, the system can also pull in past evaluated edges for queries in the same product category: "Last time an agent evaluated a firmware issue on this product line, it used kb:4821 (identical symptoms, same SKU) and rejected kb:2201 (wrong product line). That decision was correct." The agent is not just getting better candidates. It is getting guidance on how to reason about those candidates, grounded in what actually worked before.

Those two mechanisms work on different layers. The pipeline planner tightens the funnel so fewer, better candidates reach the agent. The past reasoning edges sharpen how the agent evaluates whatever reaches it. Together, they compress errors from both directions. This is different from a context graph, which stores world facts (this customer owns this product). The reasoning and retrieval graphs extend the context graph with behavioral facts: how agents reasoned, what worked, what did not. The pipeline planner and the LLM both walk all three layers.

The graph feedback loop

This is the critical shift. The reasoning and retrieval graphs turn retrieval from a static pipeline into a feedback loop. The funnel does not just narrow the data. It narrows itself. Every agent run produces decided, evaluated, and retrieval edges. Those edges inform the next pipeline plan. The scoping gets tighter. The graph paths get more precise. And because both graphs are written in the same transaction as the action, you have a causally consistent, traversable record of every decision the agent ever made, every piece of evidence it considered, and the retrieval path that produced it.

Let me walk through what this looks like concretely. On the first run for a given query type, the retrieval funnel starts with 2.8 million candidates and narrows to 14. The agent evaluates the top candidates, creating an evaluated edge for each one with a verdict and reason. It makes a decision. The decision turns out to be correct. The decided edge, the evaluated edges, and the retrieval edges all persist in the graph.

On the next similar query, both mechanisms kick in. The pipeline planner traverses past retrieval and reasoning edges. It knows the successful graph path. It knows which filter combination produced a good reduction ratio. It knows kb:2201 was rejected last time for being the wrong product line, so it excludes it from the candidate set entirely. The funnel goes from 2.8 million to 9 instead of 14. Meanwhile, the agent receives past evaluated edges as context alongside the candidates: "For similar firmware queries, kb:4821 was used because it described identical symptoms on the same SKU. kb:5540 was rejected because the issue was already resolved." The agent is not just seeing better candidates. It is seeing how a previous agent successfully reasoned about similar candidates.

After a hundred similar queries, the graph contains a rich web of evaluated edges for that query type. The planner can see, across all runs, which evidence records consistently get used versus rejected, which rejection reasons recur, and which evaluation patterns correlate with correct outcomes. The LLM, on every new run, gets the distilled reasoning patterns from the most relevant past decisions. The funnel is no longer exploring. The reasoning is no longer starting from scratch. Both layers are applying learned patterns. And the patterns are not summaries or statistics computed offline. They are live graph topology that gets traversed on every run.

This is what "deterministic" means in practice. Not that the LLM produces identical tokens every time. It means that given the same type of query, the system follows the same learned retrieval pattern, produces the same quality of context, and arrives at the same caliber of decision. The variability gets squeezed out of the data layer, which is where it was causing problems in the first place. The model can still be creative where creativity helps. But the retrieval is locked down.

The full retrieval funnel

Putting it all together, here is the complete pipeline from raw candidates to agent context, with graphs feeding back into pipeline planning.

After the agent acts, the decided, evaluated, and retrieval edges are written back, feeding into the next cycle of pipeline planning. The funnel self-tightens with every run.

Why the graphs break in a fragmented stack

There is one more thing worth making concrete, because it is easy to nod along with "you need one database" without feeling why.

Say your reasoning and retrieval edges write to Postgres, your vector embeddings live in Pinecone, and your graph relationships are in Neo4j. The pipeline planner needs to query: "for this query type, which graph paths and filter combinations correlated with correct decisions in the last 200 runs?" That query needs to join reasoning edges and retrieval edges in Postgres with graph topology in Neo4j and vector metadata in Pinecone. What transaction boundary covers all three systems? There is not one. So the planner is reading behavioral data that may or may not reflect what actually happened, referencing graph paths that may have changed since the edges were written, and correlating against outcomes that landed in a different system at a different time. The feedback loop is broken before it starts. You are building a learning system on top of eventually consistent quicksand.

That is why this architecture needs a multi-model graph database. Not because fewer databases is aesthetically nicer, but because the reasoning and retrieval edges need to live in the same graph as the entities they reference. The feedback loop is a graph traversal: walk from an agent through its decided edges, through its evaluated edges to each piece of evidence, follow the retrieval edges, compare outcomes. That only works when the reasoning and retrieval edges, the documents, the graph relationships, the vector embeddings, and the agent state all live in the same engine with the same transactional guarantees. SurrealDB handles all of this natively. This is the exact problem we designed it to solve.

Supporting research

Cognitive Architectures for Language Agents (CoALA) — Sumers, Yao, Narasimhan, Griffiths · 2024. Formalizes agent memory into episodic, semantic, and procedural types coordinated through a single decision loop. Reasoning and retrieval graphs map directly to CoALA's episodic memory: structured records of past actions that inform future decisions. The framework makes clear that scattering memory types across separate systems works against reliable agent behavior.

From Local to Global: A Graph RAG Approach — Edge, Trinh, Larson, Truitt (Microsoft Research) · 2024. Demonstrated that combining knowledge graphs with retrieval-augmented generation significantly outperforms vector-only retrieval for complex queries requiring multi-hop reasoning and structured context.

Retrieval-Augmented Generation: A Survey — Gao, Xiong, Gao et al. · 2023. Identifies consistent failure modes in RAG at scale: top-k instability, chunk blending, precision degradation. Recommends constraining the candidate set before similarity search using structural filters and graph traversal.

Graph Retrieval-Augmented Generation: A Survey — Peng, Zhu, Liu et al. · 2024. First comprehensive overview of GraphRAG, formalizing how graph-based indexing, retrieval, and generation improve accuracy and context-awareness over similarity-only approaches.

Conclusion

Near-perfect agent accuracy comes from two things working together. A retrieval pipeline that scopes candidates using structural filters, temporal constraints, graph traversal, and business rules before similarity ranking, all within a single ACID transaction. That gets you to roughly 90–95% on large corpora by eliminating the consistency gaps and retrieval noise that cause most agent failures.

And reasoning and retrieval graphs stored as edges in the same transaction as the agent's action, feeding back into the pipeline so the funnel tightens with every run. That is what closes the gap to 99%+.

Neither works if your data models are split across separate systems. The scope query cannot compose and the feedback loop breaks unless everything shares one transaction boundary. That is why we built SurrealDB.

On the first run, 2.8 million candidates narrowed to 14. A few hundred runs later, it narrows to 9, then 6, each time more precise, each time learning from the last. That is what deterministic accuracy looks like in practice: agents that do not just perform well on average, but get better with every decision they make.

References

Sumers, T.R., Yao, S., Narasimhan, K., & Griffiths, T.L. (2024). Cognitive Architectures for Language Agents. Transactions on Machine Learning Research.
Edge, D., Trinh, H., Larson, J., & Truitt, S. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130.
Gao, Y., Xiong, Y., Gao, X., et al. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997.
Peng, B., Zhu, Y., Liu, Y., et al. (2024). Graph Retrieval-Augmented Generation: A Survey. arXiv:2408.08921.