<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: SurrealDB</title>
    <description>The latest articles on DEV Community by SurrealDB (surrealdb).</description>
    <link>https://dev.to/surrealdb</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Forganization%2Fprofile_image%2F4795%2F6422ffbe-ae06-43aa-8dc1-bfb088408cc0.png</url>
      <title>DEV Community: SurrealDB</title>
      <link>https://dev.to/surrealdb</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/surrealdb"/>
    <language>en</language>
    <item>
      <title>Using SurrealDB to understand how Buffalo buffalo buffalo Buffalo buffalo</title>
      <dc:creator>Dave MacLeod</dc:creator>
      <pubDate>Thu, 09 Jul 2026 13:54:06 +0000</pubDate>
      <link>https://dev.to/surrealdb/using-surrealdb-to-understand-how-buffalo-buffalo-buffalo-buffalo-buffalo-c3l</link>
      <guid>https://dev.to/surrealdb/using-surrealdb-to-understand-how-buffalo-buffalo-buffalo-buffalo-buffalo-c3l</guid>
      <description>&lt;p&gt;&lt;em&gt;Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;That's technically a valid English sentence, one of the ones that &lt;a href="https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo_buffalo_buffalo_Buffalo_buffalo" rel="noopener noreferrer"&gt;has its own Wikipedia article&lt;/a&gt; to explain just how it works. What makes it work is that the word buffalo is also a (rarely used) verb to mean &lt;a href="https://en.wiktionary.org/w/index.php?title=buffalo&amp;amp;oldid=90796509#Translations_2" rel="noopener noreferrer"&gt;to bully or to fool someone&lt;/a&gt;. Here's an example of the verb in practice:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"If nonfiction is where you do your best writing, or your best teaching of writing, don't be &lt;em&gt;buffaloed&lt;/em&gt; into the idea that it's an inferior species."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The other thing that makes the sentence confusing is that the word buffalo can also be the plural of buffalo. Fortunately, you can also pluralise it with an s - which is what we will do in this post from here on.&lt;/p&gt;

&lt;p&gt;With a bit of rewriting and notes in parentheses, the sentence makes a &lt;em&gt;little&lt;/em&gt; more sense.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Buffaloes from Buffalo that buffaloes from Buffalo buffalo (trick) buffalo (trick) buffaloes from Buffalo.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If that still only makes just a bit of sense, don't worry - because we can demonstrate the situation using SurrealQL instead.&lt;/p&gt;

&lt;h1&gt;
  
  
  Buffaloes that buffalo Buffalo buffaloes
&lt;/h1&gt;

&lt;p&gt;To start: if we have Buffalo buffalo, that means that there should also be non-Buffalo buffalo: buffalo that don't come from the city of Buffalo. Let's make 50 of each. Two following two &lt;code&gt;CREATE&lt;/code&gt; statements will create them with a sequential ID starting from 1 and ending at 100.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;CREATE |buffalo:1..=50| SET from = "Buffalo";
CREATE |buffalo:51..=100| SET from = "Somewhere else";
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now we need to get the buffaloes to interact with each other.&lt;/p&gt;

&lt;p&gt;Since the original sentence is talking about how a buffalo can buffalo another buffalo, that must mean that this sometimes happens and sometimes does not. To demonstrate this, let's go through each buffalo, randomly choose a buffalo (with the &lt;a href="https://surrealdb.com/docs/reference/query-language/functions/database-functions/rand#randenum" rel="noopener noreferrer"&gt;rand::enum()&lt;/a&gt; function) that is not the buffalo in question (&lt;a href="https://surrealdb.com/docs/reference/query-language/functions/database-functions/array#arraycomplement" rel="noopener noreferrer"&gt;array::complement()&lt;/a&gt; will do that), and then use &lt;a href="https://surrealdb.com/docs/reference/query-language/functions/database-functions/rand#randbool" rel="noopener noreferrer"&gt;rand::bool()&lt;/a&gt; to give it a 50% chance of buffaloing that buffalo.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;FOR $buffalo IN SELECT * FROM buffalo {
    // We could do this in a single line but let's go one step at a time
    LET $all_buffalo = SELECT VALUE id FROM buffalo;
    LET $except_this_one = $all_buffalo.complement([$buffalo.id]);
    LET $other_buffalo = rand::enum($except_this_one);
    // Now that we have another buffalo, we'll flip a coin
    // to see if it gets 'buffaloed'
    IF rand::bool() {
        RELATE $buffalo-&amp;gt;buffaloes-&amp;gt;$other_buffalo;
    };
};
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With that set up, we can do a query to see which Buffalo buffaloes are buffaloing Buffalo buffaloes, and who they are. To do that, we can use a &lt;code&gt;WHERE&lt;/code&gt; clause to ensure that the buffalo in question is from Buffalo, and that the path &lt;code&gt;-&amp;gt;buffaloes-&amp;gt;buffalo.from&lt;/code&gt; (the buffalo(es) that are getting buffaloed) contains the word "Buffalo".&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;SELECT 
    id, 
    from, 
    -&amp;gt;buffaloes-&amp;gt;buffalo AS buffaloes
FROM buffalo 
WHERE 
    from = "Buffalo" 
AND 
    "Buffalo" IN -&amp;gt;buffaloes-&amp;gt;buffalo.from;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because half of the buffaloes are from Buffalo and half of these have buffaloed another buffalo, the output should show...about 25? Not quite, because some unlucky buffaloes have been buffaloed by more than one buffalo. We can demonstrate that by doing a reverse graph query on the buffaloes to see which ones they are being buffaloed by, followed by a filter to only retain the ones that have been buffaloed more than once.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;(SELECT &amp;lt;-buffaloes&amp;lt;-buffalo AS buffaloed_by FROM buffalo)
    .filter(
        |$b| $b.buffaloed_by.len() &amp;gt; 1
    );
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h1&gt;
  
  
  Buffaloing back
&lt;/h1&gt;

&lt;p&gt;Now it's time for justice to be done. Because the original sentence can actually be read as a decree.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"No buffalo from Buffalo shall be buffaloed by a buffalo from Buffalo without the chance to buffalo back. Henceforth, any buffalo from Buffalo that Buffalo buffalo buffalo shall also buffalo a Buffalo buffalo."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Sounds fair.&lt;/p&gt;

&lt;p&gt;To enact this decree, we will need to have every buffalo that is buffaloed by a buffalo from Buffalo now also buffalo another buffalo from Buffalo.&lt;/p&gt;

&lt;p&gt;To start, we'll use much of the same query as above but turn it around. And this time we won't use rand::bool() because the buffalo from Buffalo in question that has been buffaloed now has a 100% chance of buffaloing another buffalo from Buffalo.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;FOR $buffalo IN SELECT VALUE
    id
FROM buffalo 
WHERE 
    from = "Buffalo" 
AND 
    "Buffalo" IN &amp;lt;-buffaloes&amp;lt;-buffalo.from {
         LET $all_buffalo = SELECT VALUE id FROM buffalo;
         LET $except_this_one = $all_buffalo.complement([$buffalo.id]);
        LET $other_buffalo = rand::enum($except_this_one);
        RELATE $buffalo-&amp;gt;buffaloes-&amp;gt;$other_buffalo;
};
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h1&gt;
  
  
  Visualising all the buffaloing
&lt;/h1&gt;

&lt;p&gt;We can finish this off by visualising the data to see all of the buffaloing that is going on, whether actively (buffaloing a buffalo) or passively (being buffaloed by a buffalo). The following two queries will show who is buffaloing and who they are being buffaloed by, depending on whether they are from Buffalo or not.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;SELECT 
    id, 
    buffaloes-&amp;gt;buffalo AS buffaloes, 
    &amp;lt;-buffaloes&amp;lt;-buffalo AS buffaloed_by
FROM buffalo 
WHERE from = "Buffalo";

SELECT 
    id, 
    buffaloes-&amp;gt;buffalo AS buffaloes,
    &amp;lt;-buffaloes&amp;lt;-buffalo AS buffaloed_by
FROM buffalo 
WHERE from != "Buffalo";
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The raw output of these queries is easy enough to read, but it's tough to get a top-level idea of what is going on here.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[
    {
        buffaloed_by: [
            buffalo:76
        ],
        buffaloes: NONE,
        id: buffalo:3
    },
    {
        buffaloed_by: [
            buffalo:45,
            buffalo:18
        ],
        buffaloes: NONE,
        id: buffalo:4
    },
    {
        buffaloed_by: [],
        buffaloes: NONE,
        id: buffalo:5
    },
//...
]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is where Surrealist's graph visualisation comes into play. To use it, just click on the "Combined" button next to the button used to run a query and change it to "Graph". As long as a query contains the id of each record, it will be able to visually show which records are connected to which other ones.&lt;/p&gt;

&lt;p&gt;You can also click on "Stray edges" to display records that aren't connected, which is what we will do here.&lt;/p&gt;

&lt;p&gt;The output will differ every time these queries are executed and every time this functionality is used to generate a visual output in Surrealist, but for the most part you will notice that there is a lot more going on between the buffaloes from Buffalo than those from somewhere else. That makes sense, because the second RELATE operation that we used was only for, and between, buffaloes from Buffalo.&lt;/p&gt;

&lt;p&gt;The first query between buffaloes from Buffalo should look somewhat busy with connections between quite a few, and a number of stray edges floating around in the middle.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs12x8u7raohc4262r49g.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs12x8u7raohc4262r49g.png" alt=" " width="800" height="747"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The second query is a bit less so, with fewer connections between these buffaloes and a lot more stray edges.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkdfr65enmku65bvvobmj.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkdfr65enmku65bvvobmj.png" alt=" " width="800" height="747"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  Learn more
&lt;/h1&gt;

&lt;p&gt;Interested in learning more? Here are some more links and blog posts to get started.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://surrealdb.com/blog/visualising-your-data-with-surrealists-graph-view" rel="noopener noreferrer"&gt;Visualising your data with Surrealist's graph view&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://surrealdb.com/blog/enhance-your-musical-skills-with-surrealists-graph-view" rel="noopener noreferrer"&gt;Enhance your musical skills with Surrealist's graph view&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://surrealdb.com/blog/agentic-retrieval-for-structured-data-with-text-to-surql" rel="noopener noreferrer"&gt;Agentic retrieval for structured data with text to surql&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And for another technically correct mind-bending sentence, check out the page on &lt;a href="https://en.wikipedia.org/wiki/James_while_John_had_had_had_had_had_had_had_had_had_had_had_a_better_effect_on_the_teacher" rel="noopener noreferrer"&gt;James while John had had had had had had had had had had had a better effect on the teacher&lt;/a&gt;.&lt;/p&gt;

&lt;h1&gt;
  
  
  Ready to try it?
&lt;/h1&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://surrealdb.com/cloud" rel="noopener noreferrer"&gt;Create a free SurrealDB Cloud instance&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://surrealdb.com/docs" rel="noopener noreferrer"&gt;Explore the SurrealQL docs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://discord.gg/surrealdb" rel="noopener noreferrer"&gt;Join the SurrealDB Discord&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>database</category>
    </item>
    <item>
      <title>Detecting connections as they form: An introduction to streaming graph pattern matching</title>
      <dc:creator>Dave MacLeod</dc:creator>
      <pubDate>Thu, 09 Jul 2026 12:58:27 +0000</pubDate>
      <link>https://dev.to/surrealdb/detecting-connections-as-they-form-an-introduction-to-streaming-graph-pattern-matching-3n21</link>
      <guid>https://dev.to/surrealdb/detecting-connections-as-they-form-an-introduction-to-streaming-graph-pattern-matching-3n21</guid>
      <description>&lt;h1&gt;
  
  
  Detecting connections as they form: An introduction to streaming graph pattern matching
&lt;/h1&gt;

&lt;p&gt;&lt;em&gt;Originally published by &lt;a href="https://edgebound.hashnode.dev/detecting-connections-as-they-form-an-introduction-to-streaming-graph-pattern-matching" rel="noopener noreferrer"&gt;Cyril Scetbon&lt;/a&gt; on 13 April, 2026&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;In my &lt;a href="https://edgebound.hashnode.dev" rel="noopener noreferrer"&gt;&lt;strong&gt;previous posts&lt;/strong&gt;&lt;/a&gt;, I explored how &lt;strong&gt;SurrealDB&lt;/strong&gt; blends document, graph, relational, and vector models into a single engine, and how you can combine it with &lt;a href="https://edgebound.hashnode.dev/building-a-smart-knowledge-agent-with-surrealdb-and-rigrs" rel="noopener noreferrer"&gt;&lt;strong&gt;Rig.rs&lt;/strong&gt;&lt;/a&gt; to build LLM-native agents in Rust. I've written about SurrealDB more than once because I genuinely believe it's one of the most exciting databases out there right now — and the more I use it, the more I find myself reaching for it as the obvious foundation whenever I need something flexible, expressive, and Rust-native. Its graph capabilities — &lt;code&gt;RELATE&lt;/code&gt;, record IDs, graph traversals — kept nagging at me with a question: what if you could &lt;strong&gt;continuously watch a graph as it grows&lt;/strong&gt; and fire off actions the moment a meaningful pattern completes?&lt;/p&gt;

&lt;p&gt;That question sent me down a rabbit hole. This post is the first in a series where I'll walk you through the concept, the configuration I'd want for such a tool, the actual implementation, and where things could go from there.&lt;/p&gt;

&lt;p&gt;Let's start at the beginning. 🔍&lt;/p&gt;

&lt;h2&gt;
  
  
  🕸️ &lt;strong&gt;What is a labeled property graph?&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;If you've read my &lt;a href="https://edgebound.hashnode.dev/surrealdb-the-game-changing-database-youve-been-waiting-for" rel="noopener noreferrer"&gt;&lt;strong&gt;SurrealDB deep-dive&lt;/strong&gt;&lt;/a&gt;, you already know that SurrealDB natively models data as a &lt;strong&gt;labeled property graph&lt;/strong&gt;. But let's make it concrete for this series.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;labeled property graph&lt;/strong&gt; has two building blocks:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Vertices&lt;/strong&gt; — entities with a type and properties. Think &lt;code&gt;person:clint_eastwood&lt;/code&gt; with a &lt;code&gt;name&lt;/code&gt; field, or &lt;code&gt;movie:unforgiven&lt;/code&gt; with a &lt;code&gt;title&lt;/code&gt; and &lt;code&gt;year&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Edges&lt;/strong&gt; — directed, typed relationships between vertices. Think &lt;code&gt;person:clint_eastwood -&amp;gt; acted_in -&amp;gt; movie:unforgiven&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Edges are first-class citizens: they have their own type (e.g., &lt;code&gt;acted_in&lt;/code&gt;, &lt;code&gt;directed&lt;/code&gt;) and can carry their own properties (e.g., the &lt;code&gt;role&lt;/code&gt; a person played).&lt;/p&gt;

&lt;p&gt;This model is perfect for representing the kind of messy, interconnected real-world data that relational tables struggle with — social networks, supply chains, knowledge graphs, movie databases.&lt;/p&gt;

&lt;p&gt;If you want to get a feel for how expressive SurrealDB's query language gets when working with this kind of data, take a look at the &lt;a href="https://surrealdb.com/docs/surrealql/datamodel/idioms#idioms" rel="noopener noreferrer"&gt;&lt;strong&gt;idioms page&lt;/strong&gt;&lt;/a&gt; in the docs. It's a compact showcase of what SurrealQL can do — and honestly, reading through it is a big part of what got me thinking about building this in the first place.&lt;/p&gt;

&lt;h2&gt;
  
  
  ⚡ &lt;strong&gt;What is streaming graph pattern matching?&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;A &lt;strong&gt;graph pattern&lt;/strong&gt; is a template describing a subgraph you care about. For example:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Find any person who &lt;strong&gt;both acted in and directed&lt;/strong&gt; the same movie."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;In a static graph, you'd run a query once and get your results. But what if the graph is &lt;strong&gt;continuously growing&lt;/strong&gt;? Edges arrive one by one as events stream in. You want to know &lt;strong&gt;the moment&lt;/strong&gt; that pattern becomes complete — not minutes later, not after a full scan.&lt;/p&gt;

&lt;p&gt;That's &lt;strong&gt;streaming graph pattern matching&lt;/strong&gt;: detecting when a pattern is satisfied in real time, incrementally, as new edges are added.&lt;/p&gt;

&lt;p&gt;The classic use cases are things like:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Fraud detection&lt;/strong&gt; — flag the moment a money flow completes a suspicious cycle&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Recommendation engines&lt;/strong&gt; — create a "you might also like" edge the second two users share enough common favorites&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Knowledge graph enrichment&lt;/strong&gt; — derive new facts automatically as raw data arrives&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Event correlation&lt;/strong&gt; — detect that a sequence of system events matches a known failure signature&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The challenge is doing this &lt;strong&gt;without re-scanning the entire graph&lt;/strong&gt; every time a single edge lands.&lt;/p&gt;

&lt;h2&gt;
  
  
  🎬 &lt;strong&gt;A running example: the movie database&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;Throughout this series I'll use a movie dataset. It's a flat CSV file where each row is tagged with an &lt;code&gt;Entity&lt;/code&gt; type:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Entity,tmdbId,movieId,name,Work,role
Person,190,,Clint Eastwood,,
Person,192,,Morgan Freeman,,
Movie,,33,Unforgiven,,
Join,190,33,,Acting,William Munny
Join,192,33,,Acting,Ned Logan
Join,190,33,,Directing,
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A few things to note:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;Person&lt;/code&gt; rows describe people.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Movie&lt;/code&gt; rows describe films.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Join&lt;/code&gt; rows describe relationships — acting roles, directing credits — linking people to movies.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This flat structure is typical of real-world export formats. The goal is to lift it into a proper graph and then continuously match patterns against it.&lt;/p&gt;

&lt;p&gt;The pattern I want to detect: &lt;strong&gt;a person who both acted in and directed the same movie&lt;/strong&gt;. Clint Eastwood fits. Morgan Freeman does not.&lt;/p&gt;

&lt;h2&gt;
  
  
  🗂️ &lt;strong&gt;Imagining the configuration&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;If I were to design a tool for this, I'd want the entire thing controlled by a single YAML configuration file. Two top-level sections: &lt;code&gt;sources&lt;/code&gt; to describe how raw data becomes a graph, and &lt;code&gt;patterns&lt;/code&gt; to express what I'm looking for in that graph. Let me walk through both in detail.&lt;/p&gt;

&lt;h3&gt;
  
  
  Sources — loading the graph
&lt;/h3&gt;

&lt;p&gt;The &lt;code&gt;sources&lt;/code&gt; section describes where your data comes from and how to turn each row into graph elements. Each source entry points to a file and defines a &lt;code&gt;format&lt;/code&gt; block with two optional scripts: &lt;code&gt;filter&lt;/code&gt; and &lt;code&gt;load&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;For the scripting language, I'd reach for &lt;a href="https://vector.dev/docs/reference/vrl/" rel="noopener noreferrer"&gt;&lt;strong&gt;VRL&lt;/strong&gt;&lt;/a&gt; — Vector Remap Language — a safe, sandboxed expression language originally built for the Vector observability pipeline. It's expressive enough for real data wrangling, and it could be extended with a handful of custom functions to express graph operations directly.&lt;/p&gt;

&lt;p&gt;Here's what a complete &lt;code&gt;sources&lt;/code&gt; block would look like for the movie dataset:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;sources&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="c1"&gt;# --- Source 1: Load Person vertices ---&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;file&lt;/span&gt;
    &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;$movie_file&lt;/span&gt;          &lt;span class="c1"&gt;# resolved at runtime via --variable movie_file=...&lt;/span&gt;
    &lt;span class="na"&gt;format&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;filter&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;msg.Entity == "Person"&lt;/span&gt;   &lt;span class="c1"&gt;# only process rows where Entity is "Person"&lt;/span&gt;
      &lt;span class="na"&gt;load&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
        &lt;span class="s"&gt;.id = record_id!("person", [msg.tmdbId])   # build a typed record ID: person:190&lt;/span&gt;
        &lt;span class="s"&gt;.name = msg.name&lt;/span&gt;
        &lt;span class="s"&gt;+ @                            # mark this object as a vertex to be created&lt;/span&gt;

  &lt;span class="c1"&gt;# --- Source 2: Load Movie vertices ---&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;file&lt;/span&gt;
    &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;$movie_file&lt;/span&gt;
    &lt;span class="na"&gt;format&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;filter&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;msg.Entity == "Movie"&lt;/span&gt;
      &lt;span class="na"&gt;load&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
        &lt;span class="s"&gt;.id = record_id!("movie", [msg.movieId])   # e.g. movie:33&lt;/span&gt;
        &lt;span class="s"&gt;.title = msg.name&lt;/span&gt;
        &lt;span class="s"&gt;+ @&lt;/span&gt;

  &lt;span class="c1"&gt;# --- Source 3: Acting edges ---&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;file&lt;/span&gt;
    &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;$movie_file&lt;/span&gt;
    &lt;span class="na"&gt;format&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;filter&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;msg.Entity == "Join" &amp;amp;&amp;amp; msg.Work == "Acting"&lt;/span&gt;
      &lt;span class="na"&gt;load&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
        &lt;span class="s"&gt;.id = record_id!("acted_in", [msg.tmdbId, msg.movieId, msg.role])&lt;/span&gt;
        &lt;span class="s"&gt;.role = msg.role&lt;/span&gt;
        &lt;span class="s"&gt;.person.id = record_id!("person", [msg.tmdbId])&lt;/span&gt;
        &lt;span class="s"&gt;.movie.id  = record_id!("movie",  [msg.movieId])&lt;/span&gt;
        &lt;span class="s"&gt;+ @person-&amp;gt;@-&amp;gt;@movie    # create the edge; the root object becomes the edge record&lt;/span&gt;

  &lt;span class="c1"&gt;# --- Source 4: Directing edges ---&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;file&lt;/span&gt;
    &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;$movie_file&lt;/span&gt;
    &lt;span class="na"&gt;format&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;filter&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;msg.Entity == "Join" &amp;amp;&amp;amp; msg.Work == "Directing"&lt;/span&gt;
      &lt;span class="na"&gt;load&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
        &lt;span class="s"&gt;.person.id = record_id!("person", [msg.tmdbId])&lt;/span&gt;
        &lt;span class="s"&gt;.movie.id  = record_id!("movie",  [msg.movieId])&lt;/span&gt;
        &lt;span class="s"&gt;+ @person-&amp;gt;directed-&amp;gt;@movie    # anonymous edge — no edge record object&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Let me unpack each key concept.&lt;/p&gt;

&lt;h4&gt;
  
  
  &lt;code&gt;filter&lt;/code&gt; — Row-level gating
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;filter:&lt;/span&gt; &lt;span class="n"&gt;msg.Entity&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"Person"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is a VRL boolean expression evaluated against each incoming row. If it returns &lt;code&gt;false&lt;/code&gt;, the row is skipped entirely. The raw CSV row is available as &lt;code&gt;msg&lt;/code&gt;. Multiple sources can read the &lt;strong&gt;same file&lt;/strong&gt; with different filters — that's intentional. The file would be read once and fanned out to all matching sources in parallel.&lt;/p&gt;

&lt;h4&gt;
  
  
  &lt;code&gt;record_id!()&lt;/code&gt; — Typed identifiers
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="n"&gt;.id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="err"&gt;!&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"person"&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="n"&gt;msg.tmdbId&lt;/span&gt;&lt;span class="ss"&gt;])&lt;/span&gt;
&lt;span class="err"&gt;#&lt;/span&gt; &lt;span class="py"&gt;Produces:&lt;/span&gt; &lt;span class="nl"&gt;person&lt;/span&gt;&lt;span class="dl"&gt;:&lt;/span&gt;&lt;span class="m"&gt;190&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This custom VRL function would build a SurrealDB-style record ID from a table name and a list of key components. If you pass multiple components, they're combined into a composite key. The result is a strongly typed identifier like &lt;code&gt;person:190&lt;/code&gt; or &lt;code&gt;acted_in:[190, 33, "William Munny"]&lt;/code&gt;. This ensures that re-ingesting the same data is idempotent — the same row always produces the same ID.&lt;/p&gt;

&lt;h4&gt;
  
  
  &lt;code&gt;+ @&lt;/code&gt; — Creating a vertex
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="n"&gt;.id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="err"&gt;!&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"person"&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="n"&gt;msg.tmdbId&lt;/span&gt;&lt;span class="ss"&gt;])&lt;/span&gt;
&lt;span class="n"&gt;.name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;msg.name&lt;/span&gt;
&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="err"&gt;@&lt;/span&gt;          &lt;span class="err"&gt;#&lt;/span&gt; &lt;span class="s2"&gt;"upsert me as a vertex"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;@&lt;/code&gt; symbol refers to the &lt;strong&gt;current root object&lt;/strong&gt; — the thing you've been building up with &lt;code&gt;.field = value&lt;/code&gt; assignments. Prefixing it with &lt;code&gt;+&lt;/code&gt; marks it for creation in the graph.&lt;/p&gt;

&lt;p&gt;One important thing to be explicit about: &lt;strong&gt;nested objects are not pushed automatically&lt;/strong&gt;. If your root object has a &lt;code&gt;.person&lt;/code&gt; sub-object, doing &lt;code&gt;+ @&lt;/code&gt; only persists the root — &lt;code&gt;.person&lt;/code&gt; is ignored. To push a nested object as its own vertex, you have to say so explicitly with &lt;code&gt;+ @person&lt;/code&gt;. This keeps the behaviour predictable and avoids accidentally creating vertices you didn't intend.&lt;/p&gt;

&lt;p&gt;You can also push &lt;strong&gt;multiple nested objects&lt;/strong&gt; in the same script — &lt;code&gt;+ @movie&lt;/code&gt;, &lt;code&gt;+ @genre&lt;/code&gt;, and so on — each as a separate vertex, containing only its own fields.&lt;/p&gt;

&lt;h4&gt;
  
  
  &lt;code&gt;+ @from-&amp;gt;edge-&amp;gt;@to&lt;/code&gt; — Creating an edge
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="n"&gt;.person.id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="err"&gt;!&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"person"&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="n"&gt;msg.tmdbId&lt;/span&gt;&lt;span class="ss"&gt;])&lt;/span&gt;
&lt;span class="n"&gt;.movie.id&lt;/span&gt;  &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="err"&gt;!&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"movie"&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt;  &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="n"&gt;msg.movieId&lt;/span&gt;&lt;span class="ss"&gt;])&lt;/span&gt;
&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This creates a directed edge between two vertices. The vertices themselves don't have to be created in the same source entry — they can come from a different one. What matters is that &lt;code&gt;@person&lt;/code&gt; and &lt;code&gt;@movie&lt;/code&gt; resolve to objects with a valid &lt;code&gt;.id&lt;/code&gt; field at execution time.&lt;/p&gt;

&lt;p&gt;Worth knowing: an edge can be written to the database even if the vertices it points to don't exist yet. The edge record is created regardless, but any pattern that tries to traverse it will come back empty until both endpoints are actually present. This is why source order doesn't have to be strict — edges landing before their vertices are harmless, they just won't produce matches until the graph is complete enough to satisfy the pattern.&lt;/p&gt;

&lt;p&gt;If you want the &lt;strong&gt;edge itself to carry properties&lt;/strong&gt; (like a role name), you use the root object as the edge record:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="n"&gt;.id&lt;/span&gt;    &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="err"&gt;!&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"acted_in"&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="n"&gt;msg.tmdbId&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg.movieId&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg.role&lt;/span&gt;&lt;span class="ss"&gt;])&lt;/span&gt;
&lt;span class="n"&gt;.role&lt;/span&gt;  &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;msg.role&lt;/span&gt;
&lt;span class="n"&gt;.person.id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="err"&gt;!&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"person"&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="n"&gt;msg.tmdbId&lt;/span&gt;&lt;span class="ss"&gt;])&lt;/span&gt;
&lt;span class="n"&gt;.movie.id&lt;/span&gt;  &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="err"&gt;!&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"movie"&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt;  &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="n"&gt;msg.movieId&lt;/span&gt;&lt;span class="ss"&gt;])&lt;/span&gt;
&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;    &lt;span class="err"&gt;#&lt;/span&gt; &lt;span class="err"&gt;@&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;the&lt;/span&gt; &lt;span class="n"&gt;middle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"use the root object as the edge record"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Direction is explicit: &lt;code&gt;-&amp;gt;&lt;/code&gt; means left-to-right, &lt;code&gt;&amp;lt;-&lt;/code&gt; means right-to-left. You can chain them:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;acted_in&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;span class="err"&gt;#&lt;/span&gt; &lt;span class="n"&gt;Two&lt;/span&gt; &lt;span class="n"&gt;edges&lt;/span&gt; &lt;span class="n"&gt;created&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="ow"&gt;single&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Patterns — Matching the graph
&lt;/h3&gt;

&lt;p&gt;The &lt;code&gt;patterns&lt;/code&gt; section is where things get interesting. Each pattern entry has a &lt;code&gt;match&lt;/code&gt; field containing a small DSL that describes the subgraph you're looking for.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;patterns&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;match&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|-&lt;/span&gt;
      &lt;span class="s"&gt;/&lt;/span&gt;
        &lt;span class="s"&gt;m:movie&amp;lt;-acted_in&amp;lt;-p:person-&amp;gt;directed-&amp;gt;m:movie&lt;/span&gt;
        &lt;span class="s"&gt;+ p-&amp;gt;acted_and_directed-&amp;gt;m&lt;/span&gt;
      &lt;span class="s"&gt;/ -&amp;gt; m as movieId, m.title as movie, p as personId, p.name as actor&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Let me break this apart piece by piece.&lt;/p&gt;

&lt;h4&gt;
  
  
  Aliases and join conditions
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;acted_in&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;m:movie&lt;/code&gt; means: &lt;em&gt;match any vertex from the&lt;/em&gt; &lt;code&gt;movie&lt;/code&gt; &lt;em&gt;table and call it&lt;/em&gt; &lt;code&gt;m&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;p:person&lt;/code&gt; means: &lt;em&gt;match any vertex from the&lt;/em&gt; &lt;code&gt;person&lt;/code&gt; &lt;em&gt;table and call it&lt;/em&gt; &lt;code&gt;p&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;The &lt;strong&gt;same alias used in multiple places means the same vertex&lt;/strong&gt;. Here &lt;code&gt;m&lt;/code&gt; appears on both sides — so the movie that &lt;code&gt;p&lt;/code&gt; acted in must be &lt;strong&gt;the exact same movie&lt;/strong&gt; that &lt;code&gt;p&lt;/code&gt; directed. That's your join condition, expressed naturally.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This single-line form is just a convenience. You can spread the same pattern across multiple lines and the meaning is identical — each line describes one edge, and shared aliases still enforce the join. All three of these are equivalent:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;acted_in&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;acted_in&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;
&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="n"&gt;acted_in&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;-&lt;/span&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;
&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Pick the direction that reads most naturally for the relationship you're describing. The engine doesn't care — it resolves the aliases and finds the join either way.&lt;/p&gt;

&lt;p&gt;This is the heart of the pattern language. You describe structure; the engine figures out the join.&lt;/p&gt;

&lt;h4&gt;
  
  
  Actions — Deriving new edges
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;acted_and_directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When the full pattern is satisfied, this line fires. It creates a new &lt;code&gt;acted_and_directed&lt;/code&gt; edge between the matched &lt;code&gt;p&lt;/code&gt; and &lt;code&gt;m&lt;/code&gt; vertices.&lt;/p&gt;

&lt;p&gt;Duplicate prevention is built into the ID scheme: the derived edge gets a composite record ID based on the two endpoint vertices — something like &lt;code&gt;acted_and_directed:[person:190, movie:33]&lt;/code&gt;. That ID is always the same for the same pair, so re-triggering the match (say, because new edges arrived and the checker ran again) produces the exact same record rather than a second one. The operation is naturally idempotent.&lt;/p&gt;

&lt;h4&gt;
  
  
  Output projections
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="err"&gt;/&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;movieId&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="n"&gt;m.title&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;movie&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;personId&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p.name&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;actor&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After the closing &lt;code&gt;/&lt;/code&gt;, you declare what gets emitted when a match is found. You can project full vertices (&lt;code&gt;m as movieId&lt;/code&gt;) or individual fields (&lt;code&gt;m.title as movie&lt;/code&gt;). Omit this section entirely and all matched aliases come back.&lt;/p&gt;

&lt;h4&gt;
  
  
  More pattern expressiveness
&lt;/h4&gt;

&lt;p&gt;The DSL could support more than simple linear paths. A few things worth being able to express:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Multiple edge types&lt;/strong&gt; — match either &lt;code&gt;rated&lt;/code&gt; or &lt;code&gt;disliked&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rated&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="n"&gt;disliked&lt;/span&gt;&lt;span class="ss"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Wildcard edge&lt;/strong&gt; — match any edge type between two vertices:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;p1:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="err"&gt;?&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;p2:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Edge filters&lt;/strong&gt; — only match edges that satisfy a property condition:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;award&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"Oscar"&lt;/span&gt;&lt;span class="ss"&gt;]&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Vertex filters&lt;/strong&gt; — only match vertices satisfying a condition:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;birth_date&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="s2"&gt;"2000-01-01"&lt;/span&gt;&lt;span class="ss"&gt;]&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;acted_in&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Multiple independent patterns&lt;/strong&gt; in a single block — all must be satisfied for the action to fire:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="err"&gt;/&lt;/span&gt;
  &lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;acted_in&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
  &lt;span class="py"&gt;p:&lt;/span&gt;&lt;span class="n"&gt;person&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="n"&gt;movie&lt;/span&gt;
  &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;acted_and_directed&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;
&lt;span class="err"&gt;/&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;m.title&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;movie&lt;/span&gt;&lt;span class="ss"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p.name&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;actor&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔄 &lt;strong&gt;What would it look like to run this?&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;At a high level, the tool would:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Read the configuration file&lt;/strong&gt; and resolve any &lt;code&gt;$variable&lt;/code&gt; placeholders passed on the command line.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Stream data from each source into the graph&lt;/strong&gt; — filtering rows, transforming them into vertices and edges via the &lt;code&gt;load&lt;/code&gt; scripts, and persisting everything to the database.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Watch the graph for pattern completions&lt;/strong&gt; — as edges land, the tool continuously checks whether any of the declared patterns are now fully satisfied.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fire actions and print results immediately&lt;/strong&gt; — the moment a match is found, the &lt;code&gt;+&lt;/code&gt; edges are created and the projected output is flushed to stdout right away, without waiting for ingestion to finish.&lt;/p&gt;

&lt;p&gt;This last point matters: matches don't accumulate and print at the end. They appear as soon as they're found, interleaved with ongoing ingestion. If you're streaming a large dataset, you'd start seeing output long before the last row is processed.&lt;/p&gt;

&lt;p&gt;For the movie example, as soon as both the &lt;code&gt;acted_in&lt;/code&gt; and &lt;code&gt;directed&lt;/code&gt; edges involving Clint Eastwood and &lt;em&gt;Unforgiven&lt;/em&gt; are present, the pattern is satisfied and a line appears on stdout. Morgan Freeman only acted, so he never completes the pattern — no false match, no output for him.&lt;/p&gt;

&lt;p&gt;The system is also tolerant of edges arriving out of order. If a &lt;code&gt;directed&lt;/code&gt; edge lands before the corresponding &lt;code&gt;acted_in&lt;/code&gt; edge exists yet, the check is retried automatically once the missing piece shows up. You don't have to pre-sort your data or reason about arrival order.&lt;/p&gt;

&lt;p&gt;The key design intent is that you never have to write a query. You declare the shape of what you're looking for and let the tool figure out when it exists.&lt;/p&gt;

&lt;h2&gt;
  
  
  🧠 &lt;strong&gt;Why this configuration style?&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;A few deliberate design choices worth calling out:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;One file per source, multiple sources per file.&lt;/strong&gt; The same CSV can produce vertices, edges, and genre tags all at once — by duplicating the source entry with different filters. No preprocessing step required.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;VRL for transformations.&lt;/strong&gt; It's expressive enough to handle type coercions, array splits, conditional logic, and nested structures, while being safe and sandboxed. No need to invent yet another mini-language for data wrangling.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Aliases as join keys.&lt;/strong&gt; Using the same alias name to enforce vertex identity across pattern lines is a small syntax choice that turns out to be remarkably readable. The pattern says what you mean.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Declarative actions.&lt;/strong&gt; The &lt;code&gt;+&lt;/code&gt; syntax for derived edges mirrors the &lt;code&gt;+&lt;/code&gt; syntax in the &lt;code&gt;load&lt;/code&gt; scripts. Once you understand one, you understand the other.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Idempotent by construction.&lt;/strong&gt; Derived edges get a deterministic ID from their endpoints, so running the same match twice is harmless. No deduplication logic to write, no risk of polluting the graph with duplicate relationships.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Results stream, they don't batch.&lt;/strong&gt; Output is flushed to stdout the moment a match is confirmed. You can pipe the output, process it downstream, or just watch it in a terminal — it behaves like any other streaming Unix tool.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  👉 &lt;strong&gt;TL;DR&lt;/strong&gt;
&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Streaming graph pattern matching&lt;/strong&gt; is about detecting when a meaningful subgraph completes — incrementally, as edges arrive, without scanning the whole graph.&lt;/p&gt;

&lt;p&gt;A YAML config maps raw data to graph elements via VRL, and a declarative DSL expresses the patterns you care about. When a pattern matches, derived edges are created automatically.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;In the next post, I'll show you the actual implementation: the tech stack, how the pieces fit together under the hood, and what the database looks like once everything is loaded. Stay tuned. 🛠️&lt;/p&gt;

</description>
      <category>database</category>
    </item>
    <item>
      <title>Introducing Scale: SurrealDB Cloud, built for high availability and scale</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Thu, 02 Jul 2026 15:53:29 +0000</pubDate>
      <link>https://dev.to/surrealdb/introducing-scale-surrealdb-cloud-built-for-high-availability-and-scale-2ib</link>
      <guid>https://dev.to/surrealdb/introducing-scale-surrealdb-cloud-built-for-high-availability-and-scale-2ib</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://x.com/tobiemh" rel="noopener noreferrer"&gt;Tobie Morgan Hitchcock&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Today we're launching Scale, a new tier of SurrealDB Cloud built for the workloads you can't afford to have go down.&lt;/p&gt;

&lt;p&gt;Our first tier, Start, was designed for building and shipping fast. Scale is designed for what happens next: production traffic, uptime commitments, and the kind of resilience your users never notice because there are always available nodes. It's the tier for teams running SurrealDB as the scalable context layer behind real applications and AI agents in production.&lt;/p&gt;

&lt;h2&gt;
  
  
  What you get with Scale
&lt;/h2&gt;

&lt;p&gt;Scale is about one thing: keeping your database available and consistent under real-world conditions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Highly-available, fault-tolerant clusters.&lt;/strong&gt; Scale runs your database as a multi-node cluster designed to survive node and infrastructure failures without dropping writes or losing consistency. A single point of failure is no longer a single point of downtime.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Multiple availability-zone deployment.&lt;/strong&gt; Your cluster is distributed across multiple availability zones, so even the loss of an entire zone doesn't take your database with it. Traffic keeps flowing while the cluster recovers in the background.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Horizontal scale.&lt;/strong&gt; As demand grows, Scale grows with it. Add capacity by scaling out across nodes rather than being capped by the size of a single machine. Start with three nodes, and keep adding to scale your application or agent's needs.&lt;/p&gt;

&lt;p&gt;See more information about SurrealDB Cloud Scale architecture &lt;a href="https://surrealdb.com/docs/manage/cloud/architecture#scale" rel="noopener noreferrer"&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Built on SurrealDS
&lt;/h2&gt;

&lt;p&gt;Scale is powered by &lt;a href="https://surrealdb.com/platform/surrealds" rel="noopener noreferrer"&gt;SurrealDS&lt;/a&gt;, SurrealDB's distributed storage engine and the foundation that makes all of this possible.&lt;/p&gt;

&lt;p&gt;SurrealDS is a new generation distributed storage architecture, rethought from first principles. Instead of coupling storage to compute on a single box or to a proprietary cloud tier, SurrealDS embeds consensus directly in SurrealDB nodes and separates the two layers cleanly. Here's what that architecture gives you.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architecture overview
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Compute and storage separation.&lt;/strong&gt; Scale compute for QPS and storage for capacity independently, so you provision for the dimension that's actually under pressure.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;No single leader.&lt;/strong&gt; Each availability-zone node writes locally, writes scale horizontally, and transactions commit once a quorum acknowledges them.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Multi-write nodes.&lt;/strong&gt; Every write node in the cluster can accept and coordinate transactions - there's no bottleneck routing all writes through one leader.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Reduced operational overhead.&lt;/strong&gt; Consensus is embedded directly in SurrealDB nodes, eliminating external coordination services.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Fewer network dependencies.&lt;/strong&gt; A single broadcast replaces multi-hop coordination, reducing latency.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Lower latency than leader-based replication.&lt;/strong&gt; With no single leader and each AZ node writing locally, transactions avoid the extra round trips of traditional leader-based systems.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Future roadmap
&lt;/h2&gt;

&lt;p&gt;Several SurrealDS capabilities are on the near-term roadmap and will land in the Scale tier as they ship:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Object-storage backed.&lt;/strong&gt; Persist transactional data directly to commodity object storage - storage costs drop and capacity scales far beyond any single disk.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Cross-region replication.&lt;/strong&gt; Data flows through object storage rather than between nodes, significantly reducing data-transfer costs while keeping regions in sync.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Instant recovery.&lt;/strong&gt; A crashed node restores from object storage, with recovery time almost instant regardless of dataset size.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Instant branching.&lt;/strong&gt; Clone a petabyte-scale database in seconds for development, testing, or experimentation.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Get started
&lt;/h2&gt;

&lt;p&gt;If you're running SurrealDB in production, or getting ready to, Scale gives you the availability and fault-tolerance your workloads need, on the storage architecture no one else has.&lt;/p&gt;

&lt;p&gt;Explore the tier in &lt;a href="https://surrealdb.com/cloud" rel="noopener noreferrer"&gt;SurrealDB Cloud&lt;/a&gt;, and dig into the engine underneath at &lt;a href="https://surrealdb.com/platform/surrealds" rel="noopener noreferrer"&gt;SurrealDS&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>surrealdbcloud</category>
      <category>cloud</category>
      <category>database</category>
    </item>
    <item>
      <title>Bring SurrealDB to your Replit Agent</title>
      <dc:creator>itsezc</dc:creator>
      <pubDate>Thu, 02 Jul 2026 15:23:55 +0000</pubDate>
      <link>https://dev.to/surrealdb/bring-surrealdb-to-your-replit-agent-57i</link>
      <guid>https://dev.to/surrealdb/bring-surrealdb-to-your-replit-agent-57i</guid>
      <description>&lt;p&gt;Replit Agent turns a prompt into a working app. SurrealDB is the one database behind that app, covering documents, graph, vectors, and SQL in a single engine. The piece that connects them is the Model Context Protocol (MCP).&lt;/p&gt;

&lt;p&gt;Every SurrealDB instance running 3.1 or later can expose a first-party MCP server: a typed tool surface that AI agents call to inspect your schema and run queries safely. Replit Agent supports any MCP server as a connector, and adds them with a single click. Put the two together and Replit can build directly from the data you already have in SurrealDB.&lt;/p&gt;

&lt;p&gt;This is vibe coding for teams that already own their data. Instead of letting an agentic coding tool improvise a throwaway backend, you point Replit at SurrealDB and build on a database you already trust, with the agent reading your real schema as it goes.&lt;/p&gt;

&lt;p&gt;This guide is a hands-on quickstart. By the end you will have SurrealDB 3.1 running with its MCP server enabled, connected to a Replit project, and you will have prompted Replit Agent to generate a real interface on top of your data.&lt;/p&gt;

&lt;h2&gt;
  
  
  What you'll need
&lt;/h2&gt;

&lt;p&gt;You need a few things before you start, and none of them take more than a few minutes to set up.&lt;/p&gt;

&lt;p&gt;You need SurrealDB 3.1.0 or later. You also need a way to reach your instance from the internet, because Replit's connectors talk to your MCP server over HTTP and the server needs a public URL. A SurrealDB Cloud instance gives you one out of the box; for a local instance you can use a tunnel, which we cover below. You need a Replit account. Finally, you need some data. Even a handful of records is enough to see the workflow end to end, and we create a small schema below so you have something to build against.&lt;/p&gt;

&lt;p&gt;It is worth noting how this differs from Replit's built-in database. Replit Agent provisions a managed Postgres instance for the app it builds, which it owns. With SurrealDB over MCP, you own the database. Replit connects to a SurrealDB instance you already run, reads its schema as context, and can query or mutate it through a typed, permissioned tool surface. This is the right pattern when SurrealDB is your system of record and you want to build interfaces on top of it rather than spin up a new backend.&lt;/p&gt;

&lt;h2&gt;
  
  
  How SurrealDB, MCP, and Replit fit together
&lt;/h2&gt;

&lt;p&gt;There are three moving parts to understand before the steps.&lt;/p&gt;

&lt;p&gt;SurrealDB 3.1 exposes a typed tool surface for AI agents. Rather than handing an agent a raw SQL console, the MCP server presents twelve defined tools: &lt;code&gt;query&lt;/code&gt;, &lt;code&gt;select&lt;/code&gt;, &lt;code&gt;create&lt;/code&gt;, &lt;code&gt;insert&lt;/code&gt;, &lt;code&gt;upsert&lt;/code&gt;, &lt;code&gt;update&lt;/code&gt;, &lt;code&gt;delete&lt;/code&gt;, &lt;code&gt;relate&lt;/code&gt;, &lt;code&gt;info&lt;/code&gt;, &lt;code&gt;list&lt;/code&gt;, &lt;code&gt;use&lt;/code&gt;, and &lt;code&gt;run&lt;/code&gt;. Each tool carries annotations (&lt;code&gt;read_only_hint&lt;/code&gt;, &lt;code&gt;destructive_hint&lt;/code&gt;, &lt;code&gt;idempotent_hint&lt;/code&gt;) so an MCP client like Replit knows which operations are safe and which mutate data. The server also publishes self-describing schema resources at URIs like &lt;code&gt;surrealdb://schema/ns/{ns}/db/{db}/table/{table}&lt;/code&gt;, so the agent discovers your tables and fields instead of guessing.&lt;/p&gt;

&lt;p&gt;You can run that MCP server two ways. Locally, &lt;code&gt;surreal mcp&lt;/code&gt; runs as a stdio subcommand, which suits IDE integrations on your own machine. For a remote client like Replit, the server is exposed over HTTP at &lt;code&gt;/mcp&lt;/code&gt;, sitting behind SurrealDB's existing authentication middleware. We use the HTTP path here, because Replit needs to reach your server over the network.&lt;/p&gt;

&lt;p&gt;Replit Agent treats MCP servers as connectors. Replit ships a curated list of MCP servers that install in one click, and lets you add any other server, including your SurrealDB instance, from the Integrations pane. All MCP traffic passes through Replit's security scanner, which inspects tool definitions and planned executions and blocks anything it judges unsafe before it runs. Once your SurrealDB endpoint is connected, the agent pulls your live schema and data into context while it builds.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: Install or upgrade to SurrealDB 3.1
&lt;/h2&gt;

&lt;p&gt;If you are already on a 3.1.x release you can skip ahead. Otherwise, pick whichever install path matches your setup. Each pins to a specific version; drop the &lt;code&gt;--version&lt;/code&gt; flag to always get the newest stable.&lt;/p&gt;

&lt;p&gt;macOS (Homebrew), then upgrade in place:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;surreal upgrade --version 3.1.5
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Linux and macOS install script, which auto-detects your architecture:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;curl -sSf https://install.surrealdb.com | sh -s -- --version 3.1.5
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Windows (PowerShell), then upgrade in place:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;surreal upgrade --version 3.1.5
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Docker, to pull and run the exact image:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;docker run --rm --pull always -p 8000:8000 surrealdb/surrealdb:v3.1.5 start
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;surreal upgrade&lt;/code&gt; command can swap any existing install, whether Homebrew, install script, or manual binary, to the version you specify. Patch releases on the 3.1 line are drop-in upgrades.&lt;/p&gt;

&lt;p&gt;If you are on SurrealDB Cloud, you can upgrade your instance in place from the Surrealist app, and you already have a public HTTPS endpoint, which makes the networking in Step 4 considerably simpler.&lt;/p&gt;

&lt;p&gt;Verify your version:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;surreal version
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: Start your instance and create some data
&lt;/h2&gt;

&lt;p&gt;For a local run, start a server with authentication enabled. The MCP HTTP endpoint sits behind the same auth middleware as the rest of SurrealDB, so credentials matter here.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;surreal start \
  --user root --pass pass \
  rocksdb://mydata.db
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This starts SurrealDB on &lt;code&gt;http://localhost:8000&lt;/code&gt; with a persistent RocksDB store and a root user. In production you would scope down to a namespace and database user rather than root, which we cover under permissions later.&lt;/p&gt;

&lt;p&gt;Now give Replit something to build against. Use the CLI's &lt;code&gt;surreal sql&lt;/code&gt; command to open a session, or use Surrealist, the visual query tool, and define a small schema. We model a simple product catalog with reviews, which is enough to show off SurrealDB's record links.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;-- Pick a namespace and database to work in
USE NS shop DB catalog;

-- A products table
DEFINE TABLE product SCHEMAFULL;
DEFINE FIELD name        ON product TYPE string;
DEFINE FIELD price       ON product TYPE number;
DEFINE FIELD in_stock    ON product TYPE bool DEFAULT true;
DEFINE FIELD created_at  ON product TYPE datetime DEFAULT time::now();

-- A reviews table that links back to a product
DEFINE TABLE review SCHEMAFULL;
DEFINE FIELD product ON review TYPE record&amp;lt;product&amp;gt; REFERENCE;
DEFINE FIELD rating  ON review TYPE int ASSERT $value IN 0..=5;
DEFINE FIELD body    ON review TYPE string;

-- Seed a few records
CREATE product SET name = "Aeropress", price = 39.95;
CREATE product SET name = "Gooseneck Kettle", price = 64.00;
CREATE product SET name = "Burr Grinder", price = 129.00, in_stock = false;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A &lt;code&gt;SCHEMAFULL&lt;/code&gt; table means SurrealDB enforces the field definitions, which is what you want when an AI agent is going to read and write the table. The schema becomes the contract the agent builds against. Because SurrealDB publishes that schema as an MCP resource, Replit reads these field types directly rather than inferring them from sample rows.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: Enable the MCP server over HTTP
&lt;/h2&gt;

&lt;p&gt;In 3.1, the MCP HTTP surface is served at &lt;code&gt;/mcp&lt;/code&gt; on the same port as the rest of the HTTP API, behind authentication. A handful of environment variables let you tune its limits. The defaults are sensible, but it is worth knowing they exist:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;SURREAL_HTTP_MAX_MCP_BODY_SIZE&lt;/code&gt;: maximum request body size (default 4 MiB)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;SURREAL_MCP_QUERY_TIMEOUT_SECS&lt;/code&gt;: per-query timeout (default 60 s)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;SURREAL_MCP_MAX_RESULT_BYTES&lt;/code&gt;: maximum result payload (default 256 KiB)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;SURREAL_MCP_RUN_MAX_ARGS&lt;/code&gt;: maximum arguments to the run tool (default 64)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;SURREAL_MCP_PARAMS_MAX_KEYS&lt;/code&gt;: maximum bound parameters per call (default 256)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For example, to allow larger result payloads while keeping a tight query timeout:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;SURREAL_MCP_MAX_RESULT_BYTES=1048576 \
SURREAL_MCP_QUERY_TIMEOUT_SECS=30 \
surreal start --user root --pass pass rocksdb://mydata.db
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For PowerShell users on Windows:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$&lt;/span&gt;&lt;span class="nn"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="nv"&gt;SURREAL_MCP_MAX_RESULT_BYTES&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"1048576"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="nv"&gt;$&lt;/span&gt;&lt;span class="nn"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="nv"&gt;SURREAL_MCP_QUERY_TIMEOUT_SECS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"30"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;surreal&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;start&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;--user&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;root&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;--pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;rocksdb://mydata.db&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If instead you want to wire SurrealDB into a local IDE rather than Replit, run the stdio variant, &lt;code&gt;surreal mcp&lt;/code&gt;, and point your editor's MCP client at that subcommand. For Replit, stick with the HTTP endpoint.&lt;/p&gt;

&lt;p&gt;A good sanity check before involving Replit is to confirm the &lt;code&gt;/mcp&lt;/code&gt; route is reachable and that requests without valid credentials are rejected, which tells you the auth middleware is doing its job.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: Make your instance reachable
&lt;/h2&gt;

&lt;p&gt;Replit runs in the cloud, so it needs a public URL for your MCP endpoint. You have two clean options.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Option 1: SurrealDB Cloud.&lt;/strong&gt; If your data lives in a SurrealDB Cloud instance, you already have a public HTTPS endpoint with managed TLS and authentication. Your MCP URL is simply that instance's address with the &lt;code&gt;/mcp&lt;/code&gt; path appended. This is the lowest-friction path and the one we recommend for anything beyond local experimentation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Option 2: A tunnel to a local instance.&lt;/strong&gt; For local development, expose &lt;code&gt;http://localhost:8000&lt;/code&gt; through a tunneling service that gives you a temporary public HTTPS URL, such as ngrok or Cloudflare Tunnel. Your MCP URL is then &lt;code&gt;https://&amp;lt;tunnel-host&amp;gt;/mcp&lt;/code&gt;. This works well for trying things out, but treat the URL as ephemeral and never point it at production data.&lt;/p&gt;

&lt;p&gt;Either way, you end up with a single value to hand to Replit: an HTTPS URL ending in &lt;code&gt;/mcp&lt;/code&gt;, plus the credentials needed to authenticate against it.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Security note:&lt;/strong&gt; The MCP endpoint inherits SurrealDB's authentication and permissions. Before you expose anything, create a dedicated database user scoped to just the namespace and database you want Replit to touch, rather than handing over root. SurrealDB enforces record-level and field-level permissions on every query the agent runs, so a properly scoped user cannot read or write beyond what you have granted, even if the agent asks it to.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Connect SurrealDB to Replit
&lt;/h2&gt;

&lt;p&gt;Now switch over to Replit. Because MCP is a standard, you add SurrealDB the same way you would any custom server.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open your app in the Project Editor and open the Integrations pane (or go to replit.com/integrations).&lt;/li&gt;
&lt;li&gt;Choose &lt;strong&gt;Add new integration&lt;/strong&gt;, then add a custom MCP server.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Server URL:&lt;/strong&gt; paste your MCP endpoint, for example &lt;code&gt;https://your-instance.surrealdb.cloud/mcp&lt;/code&gt; or your tunnel URL ending in &lt;code&gt;/mcp&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Authentication:&lt;/strong&gt; Replit supports OAuth dynamic client registration where a server offers it, or custom headers for static tokens. For a SurrealDB instance, use a custom header with key &lt;code&gt;Authorization&lt;/code&gt; and value &lt;code&gt;Bearer &amp;lt;token&amp;gt;&lt;/code&gt; for your scoped database user. Reserve no-auth endpoints for a throwaway local demo, never for real data.&lt;/li&gt;
&lt;li&gt;Authorize the connection. Replit's security scanner inspects the server's tools, and the connection persists across your apps once approved.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Build an app on your SurrealDB data with Replit
&lt;/h2&gt;

&lt;p&gt;With the connector live, Replit Agent calls SurrealDB's tools to discover your schema and read your records, then generates UI that is wired to that data.&lt;/p&gt;

&lt;p&gt;Start by pulling your data into context with a prompt like this:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Using my SurrealDB connector, list the tables in the shop/catalog database and show me the fields on the product table.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;This prompts Replit to call the &lt;code&gt;list&lt;/code&gt; and &lt;code&gt;info&lt;/code&gt; tools and read the schema resource, then report back what it found. Behind the scenes, &lt;code&gt;use&lt;/code&gt; selects the &lt;code&gt;shop&lt;/code&gt; namespace and &lt;code&gt;catalog&lt;/code&gt; database, and the schema resource at &lt;code&gt;surrealdb://schema/ns/shop/db/catalog/table/product&lt;/code&gt; tells Replit that &lt;code&gt;product&lt;/code&gt; has &lt;code&gt;name&lt;/code&gt;, &lt;code&gt;price&lt;/code&gt;, &lt;code&gt;in_stock&lt;/code&gt;, and &lt;code&gt;created_at&lt;/code&gt; with their exact types.&lt;/p&gt;

&lt;p&gt;Once it understands your schema, ask it to build:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Build a product catalog page that shows every product from SurrealDB as a card with its name, price, and stock status. Add a filter to show only in-stock items.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Replit reads your live products via the &lt;code&gt;select&lt;/code&gt; tool, scaffolds a front end, and binds the components to the real fields. Because the agent knows &lt;code&gt;in_stock&lt;/code&gt; is a boolean from the schema, the filter it builds is correct on the first pass rather than a guess.&lt;/p&gt;

&lt;p&gt;You can go further and let it traverse relationships, which uses SurrealDB's graph capabilities:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Add a detail view for each product that lists its reviews, the rating and body, pulled from the review table that links to the product.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Here SurrealDB's record links do the heavy lifting. A SurrealQL query like &lt;code&gt;SELECT *, &amp;lt;~review.{ rating, body } AS reviews FROM product&lt;/code&gt; fetches each product together with its reviews in one round trip. The &lt;code&gt;&amp;lt;~&lt;/code&gt; operator walks incoming record references the same way graph queries do, and Replit issues exactly that through the &lt;code&gt;query&lt;/code&gt; tool.&lt;/p&gt;

&lt;p&gt;When you ask Replit to write data, such as adding a form to submit a new review, the agent reaches for the &lt;code&gt;create&lt;/code&gt; or &lt;code&gt;insert&lt;/code&gt; tool. Because those tools carry the non-read-only annotations, Replit knows they mutate state and can surface a confirmation before anything is written. Your SurrealDB permissions remain the backstop: if the connected user lacks &lt;code&gt;CREATE&lt;/code&gt; permission on &lt;code&gt;review&lt;/code&gt;, the write fails at the database regardless of what the agent attempts.&lt;/p&gt;

&lt;h2&gt;
  
  
  SurrealDB MCP tools reference
&lt;/h2&gt;

&lt;p&gt;Knowing what each tool does tells you exactly what you can ask Replit to do with your data.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;query&lt;/code&gt;&lt;/strong&gt;: run arbitrary SurrealQL. The most powerful tool, and the one behind any complex read or graph traversal.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;select&lt;/code&gt;&lt;/strong&gt;: read records from a table, optionally filtered.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;create&lt;/code&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;code&gt;insert&lt;/code&gt;&lt;/strong&gt;: add new records, with insert geared toward bulk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;upsert&lt;/code&gt;&lt;/strong&gt;: create or update depending on whether the record exists.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;update&lt;/code&gt;&lt;/strong&gt;: modify existing records.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;delete&lt;/code&gt;&lt;/strong&gt;: remove records, which is a destructive operation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;relate&lt;/code&gt;&lt;/strong&gt;: create graph edges between records, SurrealDB's relational strength.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;info&lt;/code&gt;&lt;/strong&gt;: describe the structure of a namespace, database, or table.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;list&lt;/code&gt;&lt;/strong&gt;: enumerate available namespaces, databases, or tables.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;use&lt;/code&gt;&lt;/strong&gt;: select the namespace and database to operate within.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;run&lt;/code&gt;&lt;/strong&gt;: invoke a defined function on the server.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The read-only tools (&lt;code&gt;select&lt;/code&gt;, &lt;code&gt;info&lt;/code&gt;, &lt;code&gt;list&lt;/code&gt;) are safe to let an agent call freely. The mutating tools (&lt;code&gt;create&lt;/code&gt;, &lt;code&gt;insert&lt;/code&gt;, &lt;code&gt;upsert&lt;/code&gt;, &lt;code&gt;update&lt;/code&gt;, &lt;code&gt;delete&lt;/code&gt;, &lt;code&gt;relate&lt;/code&gt;) are the ones the hint annotations flag, and the ones your database permissions should govern most tightly.&lt;/p&gt;

&lt;h2&gt;
  
  
  Production considerations for SurrealDB with Replit
&lt;/h2&gt;

&lt;p&gt;A few things are worth getting right before you let real users near a SurrealDB-backed Replit app.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Scope the connected user tightly.&lt;/strong&gt; Create a database-level user with only the permissions the app needs. SurrealDB's permission model is enforced on every query, including record-level and field-level &lt;code&gt;SELECT&lt;/code&gt;, &lt;code&gt;CREATE&lt;/code&gt;, &lt;code&gt;UPDATE&lt;/code&gt;, and &lt;code&gt;DELETE&lt;/code&gt; permissions, so the agent is constrained by the database, not just by good behavior.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Use the limits.&lt;/strong&gt; The MCP environment variables exist to keep a chatty agent from overwhelming your instance. Set a sensible &lt;code&gt;SURREAL_MCP_QUERY_TIMEOUT_SECS&lt;/code&gt; and &lt;code&gt;SURREAL_MCP_MAX_RESULT_BYTES&lt;/code&gt; for your workload.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Prefer Cloud or a stable endpoint over a dev tunnel.&lt;/strong&gt; Tunnels work for trying this out, but they are ephemeral. For anything persistent, a SurrealDB Cloud instance, or your own properly secured deployment, is the right home.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Keep your token out of shared links.&lt;/strong&gt; A Replit install link with an &lt;code&gt;Authorization&lt;/code&gt; header baked in carries a real credential. Generate it for your own use and rotate the token if it leaks.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where this leaves you
&lt;/h2&gt;

&lt;p&gt;You now have a SurrealDB instance whose schema and data are first-class context for Replit Agent, exposed through a typed, permissioned, observable MCP surface. The same connection that lets Replit build a catalog page would let it scaffold an admin dashboard, a customer-facing storefront, or an internal tool, all reading from and writing to the one database you control.&lt;/p&gt;

&lt;p&gt;That is the shape of the SurrealDB and Replit workflow: SurrealDB is the system of record and the source of truth for structure, and Replit is the interface layer that builds on top of it. MCP is the standard that makes the two speak the same language, with no bespoke integration to maintain on either side. It is what turns vibe coding and agentic coding from a demo trick into a workflow you can run against your own data.&lt;/p&gt;

&lt;p&gt;From here, a few good next steps are to define a SurrealDB function and invoke it from Replit via the &lt;code&gt;run&lt;/code&gt; tool, so business logic lives in the database where the agent can reuse it; to model a richer graph with &lt;code&gt;RELATE&lt;/code&gt; and ask Replit to build views that traverse it; and to move from a dev tunnel to a Cloud instance and tighten your user permissions for a real deployment.&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>replit</category>
      <category>mcp</category>
      <category>agents</category>
    </item>
    <item>
      <title>What's new in Surrealist 3.9</title>
      <dc:creator>Matthew McFadden</dc:creator>
      <pubDate>Tue, 23 Jun 2026 17:10:31 +0000</pubDate>
      <link>https://dev.to/surrealdb/whats-new-in-surrealist-39-2eb2</link>
      <guid>https://dev.to/surrealdb/whats-new-in-surrealist-39-2eb2</guid>
      <description>&lt;p&gt;We're excited to announce the release of Surrealist 3.9! This version introduces a complete design overhaul, a new datasets browser and data manager, an improved record inspector, and much more. Let's dive into what's new 🎉&lt;/p&gt;

&lt;h2&gt;
  
  
  Highlights
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Complete design overhaul
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fprhvvd2rexjz4b8e7sry.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fprhvvd2rexjz4b8e7sry.webp" alt="Surrealist overview page" width="800" height="478"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Surrealist 3.9 introduces the most significant visual refresh since Surrealist 2.0. From the new sidebar and topbar to the fully revamped light theme, the entire interface has been redesigned to feel fresher, cleaner, more consistent, and easier to navigate.&lt;/p&gt;

&lt;p&gt;Namespace and database selection has been unified into a single selection menu, making it faster to switch context without the need to navigate separate menus. Additionally, the new search input makes it easy to find what you need, even in large deployments.&lt;/p&gt;

&lt;p&gt;Connection and instance settings have also been completely redesigned, and the overview page now includes search for both connections and organisations. These changes make it easier to find what you are looking for no matter how many instances you manage.&lt;/p&gt;

&lt;h3&gt;
  
  
  New datasets browser and data manager
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9h38frn627jwqmkw09b8.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9h38frn627jwqmkw09b8.webp" alt="Surrealist data manager page" width="800" height="483"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Working with imports and exports of your database and trying out new features with official SurrealDB demo datasets has never been easier! Surrealist 3.9 introduces a data manager and dedicated datasets browser to make managing bulk data in your database faster and more intuitive. &lt;/p&gt;

&lt;p&gt;Additionally, it is now easier than ever to discover, browse, and apply official SurrealDB datasets to your database or Sandbox instance so that you can learn about and try new SurrealDB features in an interactive manner without needing to scroll through docs.&lt;/p&gt;

&lt;h3&gt;
  
  
  Improved record inspector
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fm6h2jhd38btzozz6pxbo.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fm6h2jhd38btzozz6pxbo.webp" alt="Surrealist record inspector" width="800" height="483"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The record inspector has been significantly enhanced to give you a deeper view of your data. The new&amp;nbsp;References&amp;nbsp;and&amp;nbsp;Live&amp;nbsp;tabs make it easier to explore record relationships and monitor real-time changes, while the new actions menu gives you more control over how you interact with individual records.&lt;/p&gt;

&lt;p&gt;These improvements make the record inspector a more capable tool for day-to-day data exploration whether you are tracing graph relationships, inspecting linked records, or watching live query results update in real time.&lt;/p&gt;

&lt;h2&gt;
  
  
  Full changelog
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Improved designer view to use ALTER statements in 3.x instances&lt;/li&gt;
&lt;li&gt;Overhauled Surrealist design

&lt;ul&gt;
&lt;li&gt;Improved brand icons&lt;/li&gt;
&lt;li&gt;Redesigned light theme&lt;/li&gt;
&lt;li&gt;Redesigned sidebar and topbar&lt;/li&gt;
&lt;li&gt;Added a toggle sidebar button to the topbar&lt;/li&gt;
&lt;li&gt;Moved settings button to a menu item in the topbar when logged out and account menu when logged in&lt;/li&gt;
&lt;li&gt;Unified the namespace and database selectors into a single selection menu&lt;/li&gt;
&lt;li&gt;Completely redesigned connection and instance settings&lt;/li&gt;
&lt;li&gt;Implemented new search functionality for connections and organisations on the overview page&lt;/li&gt;
&lt;li&gt;Revamped and re-enabled API Docs view&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Implemented a new datasets browser and data manager (&lt;a href="https://github.com/surrealdb/surrealist/issues/1224" rel="noopener noreferrer"&gt;#1224&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Improved the record inspector with new references and live tabs, and new actions&lt;/li&gt;
&lt;li&gt;Implemented VIM mode setting (&lt;a href="https://github.com/surrealdb/surrealist/issues/106" rel="noopener noreferrer"&gt;#106&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Implemented the ability to toggle tables in the designer (&lt;a href="https://github.com/surrealdb/surrealist/issues/372" rel="noopener noreferrer"&gt;#372&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Implemented default namespace and database functionality for 3.x instances&lt;/li&gt;
&lt;li&gt;Implemented namespace and database comment editor&lt;/li&gt;
&lt;li&gt;Added namespace and database comments to selection menu and database settings page&lt;/li&gt;
&lt;li&gt;Added search inputs to the namespace and database selection lists&lt;/li&gt;
&lt;li&gt;Added designer view LOD settings for zoom-based table simplification (&lt;a href="https://github.com/surrealdb/surrealist/issues/1222" rel="noopener noreferrer"&gt;#1222&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Added pagination to the newsletter drawer&lt;/li&gt;
&lt;li&gt;Fixed title bar overlapping drawers on Windows and Linux&lt;/li&gt;
&lt;li&gt;Fixed authentication redirect on session expiry&lt;/li&gt;
&lt;li&gt;Fixed Windows install art not showing for new installations&lt;/li&gt;
&lt;li&gt;Fixed Surrealist mini not allowing datasets other than surreal-deal-store&lt;/li&gt;
&lt;li&gt;Fixed an error where imports always showed success, even when they failed&lt;/li&gt;
&lt;li&gt;Fixed 2.x sample data datasets being applied to 3.x databases&lt;/li&gt;
&lt;li&gt;Fixed explorer returning no rows when sorting on empty columns (&lt;a href="https://github.com/surrealdb/surrealist/issues/1194" rel="noopener noreferrer"&gt;#1194&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed interface and designer zoom not applying on all platforms (&lt;a href="https://github.com/surrealdb/surrealist/issues/1188" rel="noopener noreferrer"&gt;#1188&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Clean up stale graph connections after deleting edge tables (&lt;a href="https://github.com/surrealdb/surrealist/issues/1147" rel="noopener noreferrer"&gt;#1147&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed an issue where clicking a table in designer view would cause a crash (&lt;a href="https://github.com/surrealdb/surrealist/issues/1238" rel="noopener noreferrer"&gt;#1238&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed an issue causing large conection urls to cause UI overflow in connection cards (&lt;a href="https://github.com/surrealdb/surrealist/issues/1225" rel="noopener noreferrer"&gt;#1225&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed broken documentation links (&lt;a href="https://github.com/surrealdb/surrealist/issues/1217" rel="noopener noreferrer"&gt;#1217&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed passing URLs through desktop launcher (&lt;a href="https://github.com/surrealdb/surrealist/pull/1179" rel="noopener noreferrer"&gt;#1179&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed Designer exports including lower LOD levels&lt;/li&gt;
&lt;li&gt;Fixed designer not showing links for complex ids (&lt;a href="https://github.com/surrealdb/surrealist/issues/1063" rel="noopener noreferrer"&gt;#1063&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed UUID records not resolving correctly in graph (&lt;a href="https://github.com/surrealdb/surrealist/issues/974" rel="noopener noreferrer"&gt;#974&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed clipboard writes failing in some browsers (&lt;a href="https://github.com/surrealdb/surrealist/issues/994" rel="noopener noreferrer"&gt;#994&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Fixed GraphQL view authentication (&lt;a href="https://github.com/surrealdb/surrealist/issues/1101" rel="noopener noreferrer"&gt;#1101&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We hope you enjoy these new features and improvements! As always, we appreciate your feedback and suggestions for future releases. &lt;a href="https://discord.com/invite/surrealdb" rel="noopener noreferrer"&gt;Join the SurrealDB Discord&lt;/a&gt; to engage with the community and receive support.&lt;/p&gt;

&lt;p&gt;Get started for free today at &lt;a href="https://app.surrealdb.com/c/sandbox/query" rel="noopener noreferrer"&gt;app.surrealdb.com&lt;/a&gt;&lt;/p&gt;

</description>
      <category>database</category>
      <category>surrealdb</category>
      <category>news</category>
      <category>web</category>
    </item>
    <item>
      <title>SurrealDB is now available on the Nebius AI Cloud Marketplace</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Thu, 18 Jun 2026 15:33:29 +0000</pubDate>
      <link>https://dev.to/surrealdb/surrealdb-is-now-available-on-the-nebius-ai-cloud-marketplace-34fj</link>
      <guid>https://dev.to/surrealdb/surrealdb-is-now-available-on-the-nebius-ai-cloud-marketplace-34fj</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://x.com/tobiemh" rel="noopener noreferrer"&gt;Tobie Morgan Hitchcock&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We're excited to announce that SurrealDB is now officially available on the Nebius AI Cloud Marketplace.&lt;/p&gt;

&lt;p&gt;This launch reflects our commitment to meeting developers and AI teams where they build, and deepens our partnership with Nebius to make SurrealDB easier to adopt, deploy, and scale on one of the fastest-growing AI cloud platforms in the world.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Nebius?
&lt;/h2&gt;

&lt;p&gt;Nebius is a full-stack AI cloud built for the demands of modern AI workloads. Powered by the latest GPU and high-performance networking infrastructure, with managed platform, serverless AI, and storage built for the entire AI lifecycle, Nebius gives teams the infrastructure to train, tune, and serve models at scale.&lt;/p&gt;

&lt;p&gt;As an NVIDIA Reference Platform Cloud Partner with infrastructure across Europe and the US, Nebius pairs bare-metal performance with the operational simplicity and cost efficiency that AI teams need to move from experiment to production. It's a natural home for SurrealDB - giving you a unified data layer right alongside the compute powering your AI and agentic applications.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why SurrealDB?
&lt;/h2&gt;

&lt;p&gt;SurrealDB combines document, graph, and relational capabilities in one unified database - designed for agent memory, context layers, modern applications, real-time systems, and AI-powered workloads.With flexible schema, SurrealQL, and built-in real-time features, SurrealDB enables you to build semantic and context layers powered by knowledge graphs for agent memory and multi-agentic workflows. &lt;/p&gt;

&lt;p&gt;SurrealDB simplifies your technology stack, reduces Total Cost of Ownership and operational complexity, and allows you to ship products and features in days rather than weeks.&lt;/p&gt;

&lt;h2&gt;
  
  
  Get started
&lt;/h2&gt;

&lt;p&gt;SurrealDB is available now on the &lt;a href="https://console.nebius.com/project-e00j0yxxpr003gvmj7t4vd/applications/surrealdb" rel="noopener noreferrer"&gt;Nebius AI Cloud Marketplace&lt;/a&gt;. We can't wait to see what you build, join our &lt;a href="https://discord.com/invite/surrealdb" rel="noopener noreferrer"&gt;Discord community&lt;/a&gt; and let us know how you get on!&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>database</category>
      <category>nebius</category>
      <category>ai</category>
    </item>
    <item>
      <title>SurrealDB 3.x by the numbers</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Fri, 29 May 2026 19:20:49 +0000</pubDate>
      <link>https://dev.to/surrealdb/surrealdb-3x-by-the-numbers-39ao</link>
      <guid>https://dev.to/surrealdb/surrealdb-3x-by-the-numbers-39ao</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://x.com/tobiemh" rel="noopener noreferrer"&gt;Tobie Morgan Hitchcock&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  One engine, multi-workloads, full durability.
&lt;/h2&gt;

&lt;p&gt;You can explore the full results, methodology, and per-database breakdowns at &lt;a href="https://surrealdb.com/benchmarks" rel="noopener noreferrer"&gt;surrealdb.com/benchmarks&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why we ran these
&lt;/h2&gt;

&lt;p&gt;Database benchmarks are notoriously easy to game, and difficult to get right. Different hardware, different durability settings, different client libraries, a workload that happens to suit one engine's indexing strategy - any of those will tilt the numbers. So we did three things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Ran every database on the same hardware - an AMD Ryzen Threadripper 9970X (32C/64T), 128 GiB DDR5, NVMe storage, Ubuntu 24.04.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Used the same open-source harness - crud-bench - with each workload translated into each engine's native query language so no database is penalised for an unfamiliar dialect.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Configured every engine for production-grade durability - fsync on, snapshot isolation, no in-memory shortcuts (except where explicitly noted for embedded comparisons).&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;We also went out of our way to give every database a fair shot. Rather than running each engine on its out-of-the-box defaults, we used&amp;nbsp;&lt;strong&gt;optimised configurations across the board&lt;/strong&gt;&amp;nbsp;- the same kind of tuning a production team would apply before going live. That meant raising connection and worker pool limits to match the 128-client load, sizing buffer pools, page caches, and shared memory to take advantage of the available 128 GiB of RAM, enabling parallel query execution and prepared-statement caching where supported, setting WAL and checkpoint intervals to values recommended by each project's own performance guides, and turning on the indexes and storage engines (InnoDB, WiredTiger, RocksDB-backed stores, etc.) that each database's documentation recommends for OLTP workloads. The goal was to make sure no engine was held back by a conservative default - if a database underperforms here, it isn't because we left it on its laptop-friendly starter config.&lt;/p&gt;

&lt;p&gt;Workloads run with 128 clients issuing 48 concurrent queries each, against datasets of a single table with 5 - 15 million rows of mixed-type records (strings, integers, floats, UUIDs, datetimes, booleans, large text fields, geospatial data, and nested objects and arrays).&lt;/p&gt;

&lt;h2&gt;
  
  
  About last time
&lt;/h2&gt;

&lt;p&gt;We owe a word on durability. The previous round of benchmark results ran with&amp;nbsp;&lt;code&gt;fsync&lt;/code&gt;&amp;nbsp;disabled for every engine - leaving each database's writes in the OS page cache rather than flushed to disk. Every database in the comparison ran with the same setting, so nothing was being "fudged" relative to the other engines, but we didn't make the setting explicit, and the headline numbers ended up describing a workload that most production deployments would not likely run.&lt;/p&gt;

&lt;p&gt;This round is different.&amp;nbsp;&lt;strong&gt;Every database in these benchmarks runs with full disk durability enabled&lt;/strong&gt;&amp;nbsp;-&amp;nbsp;&lt;code&gt;fsync&lt;/code&gt;&amp;nbsp;on, WAL flushed on every commit, no buffered writes hiding behind the page cache. The configuration files for each engine are checked into the &lt;a href="https://github.com/surrealdb/crud-bench" rel="noopener noreferrer"&gt;crud-bench repository&lt;/a&gt;&amp;nbsp;so anyone can audit them. The numbers above are what each engine sustains when every committed transaction is on disk before the client gets an acknowledgement. That's slower than the cache-friendly numbers you'll find in some marketing posts, ours included, but it's the only honest way to compare databases that are going to outlive a power outage.&lt;/p&gt;

&lt;h2&gt;
  
  
  &amp;nbsp;How far SurrealDB has come
&lt;/h2&gt;

&lt;p&gt;The biggest story is internal. Across three major releases, we've fundamentally rebuilt the query, parser, and storage layers:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fo8rdhc9mhnypvtd651en.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fo8rdhc9mhnypvtd651en.png" alt="how far SurrealDB has come" width="800" height="170"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Between SurrealDB 2.x and 3.x alone:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;31% faster mean CRUD throughput&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;58% faster batch operations&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;11,894% faster non-indexed full-table scans&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;136% faster indexed queries&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Tail latency improvements of 27% (CRUD), 32% (batches), 59% (indexed), and 99% (scans)&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The scan number is not a typo. The SurrealDB 3.x query planner and storage engine eliminates the per-row decoding overhead that dominated earlier versions, which is why a workload that used to take minutes now completes in seconds.&lt;/p&gt;

&lt;h2&gt;
  
  
  How we compare to other databases
&lt;/h2&gt;

&lt;p&gt;SurrealDB is a durable, transactional, multi-model database, so the comparisons that matter most are against the primary databases people actually evaluate it against - Postgres for relational, MongoDB for document, Neo4j for graph. Here's how the same workload looks across those three categories, run on the same hardware with each engine on a tuned production-grade configuration.&lt;/p&gt;

&lt;h3&gt;
  
  
  vs. PostgreSQL (and MySQL)
&lt;/h3&gt;

&lt;p&gt;CRUD throughput (ops/s) and bulk-read metrics:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fsrc8zkre8pfllgqwdrhp.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fsrc8zkre8pfllgqwdrhp.png" alt="vs. PostgreSQL and MySQL" width="800" height="285"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;SurrealDB is faster than Postgres on every write operation - roughly&amp;nbsp;&lt;strong&gt;1.5× faster creates, 1.3× faster updates, and 1.8× faster deletes&lt;/strong&gt;&amp;nbsp;- while Postgres still edges ahead on raw single-record reads. Against MySQL the gap widens dramatically: SurrealDB is&amp;nbsp;&lt;strong&gt;5 - 7× faster on writes&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Averaged across creates, updates, and deletes,&amp;nbsp;&lt;strong&gt;SurrealDB delivers ~1.5× the write throughput of Postgres&lt;/strong&gt;&amp;nbsp;- the headline number the benchmarks page leads the Relational category with - and beats Postgres by ~1.5× on full-table counts. Postgres' query planner is 30 years old and still ahead on indexed predicate filtering; we're not pretending otherwise, and we're working on closing that gap in 3.1.&lt;/p&gt;

&lt;h3&gt;
  
  
  vs. MongoDB (and ArangoDB)
&lt;/h3&gt;

&lt;p&gt;CRUD throughput (ops/s) and bulk-read metrics:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fammier1thf3i9smmihcx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fammier1thf3i9smmihcx.png" alt="vs. MongoDB and ArangoDB" width="800" height="235"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This is the closest race. MongoDB still leads on single-record writes, while SurrealDB is&amp;nbsp;&lt;strong&gt;~1.3× faster on reads&lt;/strong&gt;&amp;nbsp;- and the picture flips on heavier workloads. On predicate filter scans against unindexed tables - the headline figure the benchmarks page uses for the Document category -&amp;nbsp;&lt;strong&gt;SurrealDB is roughly 2.7× faster than MongoDB&lt;/strong&gt;, with consistently lower mean and p99 latency. Against ArangoDB's document engine, SurrealDB is between&amp;nbsp;&lt;strong&gt;100× and 150× faster on writes&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  vs. Neo4j (and ArangoDB)
&lt;/h3&gt;

&lt;p&gt;CRUD throughput (ops/s) and bulk-read metrics:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4s2ttjs7y6alh38f9nev.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4s2ttjs7y6alh38f9nev.png" alt="vs. Neo4j and ArangoDB" width="800" height="260"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;SurrealDB outperforms Neo4j across every CRUD operation -&amp;nbsp;&lt;strong&gt;roughly 2 - 3.5× faster&amp;nbsp;on writes&lt;/strong&gt; and&amp;nbsp;&lt;strong&gt;1.5× faster on reads&lt;/strong&gt;&amp;nbsp;- while running the same graph traversals through the same engine that handles documents and tables.&lt;/p&gt;

&lt;p&gt;The gap on filtered scans is even more dramatic. Across indexed predicate filter queries - the headline metric the benchmarks page uses for the Graph category -&amp;nbsp;&lt;strong&gt;SurrealDB is roughly 35× faster than Neo4j&lt;/strong&gt;. No separate database, no separate query language, no separate operational story.&lt;/p&gt;

&lt;h3&gt;
  
  
  Reference point: Redis and KeyDB
&lt;/h3&gt;

&lt;p&gt;For raw key-value throughput, we also ran SurrealDB's in-memory engine (with append-only persistence) against Redis and KeyDB:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvy0ei2goby1x5h5mza5f.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvy0ei2goby1x5h5mza5f.png" alt="Redis and KeyDB" width="798" height="197"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That's roughly&amp;nbsp;&lt;strong&gt;3× faster than Redis on writes, updates, and deletes&lt;/strong&gt;, while offering durable, snapshot-isolated transactions and a full query language Redis doesn't have. Redis still wins on large 1,000-record batch operations and edges ahead on single-record reads.&lt;/p&gt;

&lt;h3&gt;
  
  
  Embedded mode (vs. SQLite)
&lt;/h3&gt;

&lt;p&gt;SurrealDB's embedded engine runs the same SurrealQL on the same disk format as the server.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F87t5kfwba1xj02u5v5cc.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F87t5kfwba1xj02u5v5cc.png" alt="vs. SQLite" width="799" height="273"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Against SQLite, SurrealDB is roughly&amp;nbsp;&lt;strong&gt;85× faster on creates, 110× faster on updates, and 75× faster on deletes&lt;/strong&gt;, with single-record reads in the same ballpark. On predicate filter scans against unindexed tables it's around&amp;nbsp;&lt;strong&gt;6.5× faster&lt;/strong&gt;&amp;nbsp;than SQLite, and on indexed filter scans, around&amp;nbsp;3× faster.&lt;/p&gt;

&lt;p&gt;The full breakdowns - including p50, p95, and p99 latencies, batch sizes from 100 to 1,000 rows, indexed and non-indexed predicate filters, and full-text search - are on the &lt;a href="https://surrealdb.com/benchmarks" rel="noopener noreferrer"&gt;benchmarks page&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Closing the remaining gaps
&lt;/h2&gt;

&lt;p&gt;The numbers above are an honest snapshot, not a finish line. There are still workloads where Redis, MongoDB, and Postgres beat us - large batch operations vs. Redis, single-record writes vs. Mongo, indexed predicate filtering vs. Postgres - and we know exactly where each gap comes from. Closing those gaps is the central focus of the SurrealDB 3.1 cycle.&lt;/p&gt;

&lt;p&gt;What we're actively working on:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Batch path rewrites to bring 100-row and 1,000-row batched ops closer to Redis throughput, including better client-side pipelining and a leaner server-side batch executor.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A smarter query planner with cost-based optimisation, predicate pushdown into the storage engine, and richer index selectivity statistics - the work that gets us to parity with Postgres on indexed filter scans.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Storage layer improvements for the document workload - more compact in-place updates, sharper write amplification, and tighter integration between the key encoding and the document path resolver - which is where Mongo currently has the edge on single-record writes.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Vector and graph traversal optimisations as those workloads land in the benchmark suite, so the multi-model story holds up at the same rigour as CRUD.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The goal isn't "fastest at one thing." It's to be&amp;nbsp;&lt;strong&gt;fastest, or competitive, across every workload that matters&lt;/strong&gt;, while keeping the one property no specialist engine can match: a single query language - SurrealQL - that handles relational, document, graph, key-value, time-series, vector, and full-text search data in the same database, with the same transactional guarantees, on the same disk format - and the same engine, whether you're running it embedded inside an application, on a single server, at the edge close to your users, or distributed across hundreds of nodes for horizontal scale. We don't think you should have to choose between Postgres, Mongo, Neo4j, and Redis - or between a database that runs on a developer laptop and one that runs across a global fleet. We think a single database should run all four shapes of workload at production speed, anywhere it needs to live, and the SurrealDB 3.x numbers above are the strongest evidence yet that it can.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why this matters for AI agents
&lt;/h3&gt;

&lt;p&gt;There's a reason we keep pushing on this combination of data models, and it's not historical accident.&amp;nbsp;&lt;strong&gt;Agent memory is multi-model by nature&lt;/strong&gt;.&amp;nbsp;A useful AI agent needs structured facts about the world (relational), semi-structured context and tool outputs (document), entity and event relationships (graph), embeddings for semantic recall (vector), keyword and BM25 retrieval over its corpus (full-text search), episodic and temporal context (time-series), and fast session and cache state (key-value) - and it needs all of that in a single transactionally consistent store, because the moment those shapes live in separate databases, you've built a glue-code problem that breaks every time the schema changes or the model is updated.&lt;/p&gt;

&lt;p&gt;Agents also need that memory&amp;nbsp;&lt;strong&gt;close to where they run&lt;/strong&gt;. An agent reasoning inside a browser, a phone, an in-vehicle system, or a per-tenant edge worker can't afford a round trip to a central database for every retrieval. The fact that SurrealDB runs as an embedded engine on the same disk format as the distributed server - with the same query language and the same transactional guarantees - is what makes it usable as the memory layer for agents that move fluidly between local, edge, and centralised deployments. Faster CRUD, faster scans, and faster indexed lookups aren't just abstract benchmark wins; they're how many tools an agent can call, how much context it can recall per turn, and how many concurrent agents a single host can support. That's the workload SurrealDB 3.x is built for, and it's the workload the next round of benchmarks - covering vector search, graph traversal, and full-text retrieval - will measure head-on.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's next
&lt;/h2&gt;

&lt;p&gt;We intend to expand the benchmarks to also cover&amp;nbsp;&lt;strong&gt;CockroachDB, TiDB, MongoDB, and Aerospike&lt;/strong&gt;&amp;nbsp;for distributed comparisons, which we'll publish in a future round. We also plan to expand the workload set to include graph traversals vector search, and full-text search, two areas where the single-engine, multi-model design of SurrealDB shows its biggest advantages.&lt;/p&gt;

&lt;p&gt;Until then: the harness is open source, the results are reproducible, and we'd love for you to run them on your own hardware and tell us what you find.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://surrealdb.com/benchmarks" rel="noopener noreferrer"&gt;See the full benchmarks&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>database</category>
      <category>benchmarks</category>
      <category>news</category>
    </item>
    <item>
      <title>SurrealDB 3.1: stability, DiskANN, and a new release process</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Thu, 28 May 2026 08:43:39 +0000</pubDate>
      <link>https://dev.to/surrealdb/surrealdb-31-stability-diskann-and-a-new-release-process-20cj</link>
      <guid>https://dev.to/surrealdb/surrealdb-31-stability-diskann-and-a-new-release-process-20cj</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://github.com/tobiemh" rel="noopener noreferrer"&gt;Tobie Morgan Hitchcock&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Three months after 3.0 went GA, we're excited to announce that SurrealDB 3.1 is now available. This is the first minor release in the 3.x series. It builds on the foundations we shipped in 3.0 with a focus on stability, a second approximate-nearest-neighbour index in DiskANN, and a substantial round of security hardening. Alongside the release, we're also rolling out a change to how we develop and ship SurrealDB.&lt;/p&gt;

&lt;p&gt;The full list of changes is in the &lt;a href="https://surrealdb.com/releases/3.1.0" rel="noopener noreferrer"&gt;3.1 release notes&lt;/a&gt;. Below are the highlights.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's new in 3.1
&lt;/h2&gt;

&lt;h3&gt;
  
  
  DiskANN approximate-nearest neighbour index
&lt;/h3&gt;

&lt;p&gt;SurrealDB now ships DiskANN as a second ANN index type, sitting alongside HNSW. DiskANN trades a different set of memory and recall characteristics and is well suited to larger-than-memory vector workloads, which has been a recurring request from teams building production agent and search systems on SurrealDB.&lt;/p&gt;

&lt;p&gt;The introduction of DiskANN also drove an end-to-end overhaul of the ANN warm-lookup path. Both HNSW and DiskANN benefit from this work, with measurable improvements to warm-cache latency on the existing HNSW index for free.&lt;/p&gt;

&lt;p&gt;You can pick the index type at definition time with the new &lt;code&gt;DEFINE INDEX ... DISKANN&lt;/code&gt; syntax. See the release notes for the full set of options.&lt;/p&gt;

&lt;h3&gt;
  
  
  GraphQL: aliases, cursor pagination, and multi-model filtering
&lt;/h3&gt;

&lt;p&gt;SurrealDB v3.1 brings a significant upgrade to the GraphQL surface, making it more expressive, more predictable, and easier to integrate with modern GraphQL clients.&lt;/p&gt;

&lt;p&gt;The generated schema now follows Apollo conventions by default, with singular fetch, plural list queries, and &lt;code&gt;createX&lt;/code&gt; / &lt;code&gt;updateX&lt;/code&gt; / &lt;code&gt;deleteX&lt;/code&gt; mutations, so you always know what to expect without consulting per-database config. Fields and tables can now carry &lt;code&gt;GRAPHQL_ALIAS&lt;/code&gt; and &lt;code&gt;GRAPHQL_DEPRECATED&lt;/code&gt; clauses directly in their definitions, letting you decouple your SurrealQL identifiers from the names your API consumers see:&lt;/p&gt;

&lt;p&gt;Schema&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="n"&gt;FIELD&lt;/span&gt; &lt;span class="n"&gt;first_name&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;person&lt;/span&gt; &lt;span class="k"&gt;TYPE&lt;/span&gt; &lt;span class="n"&gt;string&lt;/span&gt;
  &lt;span class="n"&gt;GRAPHQL_ALIAS&lt;/span&gt; &lt;span class="nv"&gt;"firstName"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="n"&gt;FIELD&lt;/span&gt; &lt;span class="n"&gt;legacy_score&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;player&lt;/span&gt; &lt;span class="k"&gt;TYPE&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;
  &lt;span class="n"&gt;GRAPHQL_DEPRECATED&lt;/span&gt; &lt;span class="nv"&gt;"Use `rank` instead"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Cursor pagination arrives in this release too. Each table now gets a &lt;code&gt;&amp;lt;plural&amp;gt;Connection&lt;/code&gt; query returning &lt;code&gt;edges&lt;/code&gt;, &lt;code&gt;pageInfo&lt;/code&gt;, and a lazily-evaluated &lt;code&gt;totalCount&lt;/code&gt;, so you only pay for the count when you ask for it. Offset pagination via &lt;code&gt;limit&lt;/code&gt; and &lt;code&gt;start&lt;/code&gt; remains fully supported alongside it.&lt;/p&gt;

&lt;p&gt;On the multi-model side, GraphQL queries can now reach across SurrealDB's full data model. Full-text search, vector similarity, and time-series aggregation are all directly queryable through the GraphQL API, unlocking AI-native and analytics-heavy workloads without touching SurrealQL.&lt;/p&gt;

&lt;p&gt;This release also closes a range of reported issues: stale schema caches after DDL changes, invalid filter identifiers on nested array fields, missing &lt;code&gt;id&lt;/code&gt; range and &lt;code&gt;in&lt;/code&gt; filters, and &lt;code&gt;count()&lt;/code&gt;-based predicates in &lt;code&gt;WHERE&lt;/code&gt; clauses, all resolved and covered by integration tests.&lt;/p&gt;

&lt;h3&gt;
  
  
  Improved in-memory performance
&lt;/h3&gt;

&lt;p&gt;SurrealDB's in-memory backend now uses a datastore based on optimistic lock coupling. This improves performance by allowing readers to acquire lock-free access, only retrying at the end if a modification has occurred in the meantime. This allows readers to proceed without blocking writers, and vice versa.&lt;/p&gt;

&lt;h3&gt;
  
  
  Stability and correctness
&lt;/h3&gt;

&lt;p&gt;The bulk of the 3.1 work is bug fixes and stabilisation of the 3.0 line. This continues the stream of fixes that shipped in v3.0.1 through v3.0.5, with a substantial number of additional issues addressed across the query engine, indexing, and storage layers. If you upgraded from 2.x to 3.0 and hit any rough edges, there's a strong chance the relevant fix is in 3.1.&lt;/p&gt;

&lt;h3&gt;
  
  
  Security hardening
&lt;/h3&gt;

&lt;p&gt;3.1 closes a batch of previously reported security vulnerabilities, alongside a set of previously undiscovered issues that we found internally. The undiscovered ones come from an increased investment in LLM-assisted security review, in line with broader industry practice. Mozilla wrote a good summary of this approach recently. The combined result is a notably larger security section in this release than in previous ones.&lt;/p&gt;

&lt;p&gt;The full security section is itemised in the release notes.&lt;/p&gt;

&lt;h3&gt;
  
  
  Enterprise: audit logging and slow-query pipeline
&lt;/h3&gt;

&lt;p&gt;Customers on the Enterprise tier get two new operational tools in 3.1:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;A structured audit log capturing authentication events, schema changes, and other actions of interest to compliance and security teams.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A slow-query pipeline that surfaces queries exceeding a configurable threshold, complete with their plan and execution metadata, to help diagnose performance issues in production.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both have been on Enterprise customer wishlists for a while, and we're pleased to have them shipping in 3.1.&lt;/p&gt;

&lt;h3&gt;
  
  
  Other quality of life additions
&lt;/h3&gt;

&lt;p&gt;Among the many changes mentioned in the 3.1 release notes, some of the changes that you won't want to miss are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;A built-in MCP server for AI tools and IDEs,Unified OpenTelemetry metrics and logging pipeline,&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Unified OpenTelemetry metrics and logging pipeline,W3C trace propagation across HTTP and WebSocket,&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;W3C trace propagation across HTTP and WebSocket,Full ALTER coverage for every DEFINE statement,&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Full ALTER coverage for every DEFINE statement,&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;And continued executor and index improvements beyond the in-memory engine.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  A new release process
&lt;/h3&gt;

&lt;p&gt;3.1 is also the first release we've shipped under a new development workflow, and we want to be transparent about what's changed and why.&lt;/p&gt;

&lt;p&gt;For most of SurrealDB's history, we've developed in public on the main &lt;code&gt;surrealdb/surrealdb&lt;/code&gt; repository. When a security vulnerability needed addressing, we'd cut a temporary private fork, fix the issue there, cut a release, patch the SurrealDB Cloud fleet, and then make the release public. The intent was good. Development happened where the community could see it, and security fixes were embargoed only as long as they needed to be. In practice, though, this added real overhead to every security response. Cutting a fresh private fork for each issue, keeping it in sync with main, and merging the fix back when going public all took time we'd rather have spent on the fix itself.&lt;/p&gt;

&lt;p&gt;Starting with 3.1, we've moved all development to a private repository. The public &lt;code&gt;surrealdb/surrealdb&lt;/code&gt; repo remains the source of truth for releases, issues, and the source code that users read. Day-to-day commits now happen privately.&lt;/p&gt;

&lt;p&gt;The cadence looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;We develop and tag releases in the private repository.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;When a release is ready, we announce it publicly and ship binaries and Docker images on the usual schedule.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Roughly 1 week after each release, we sync the private repository back to the public &lt;code&gt;surrealdb/surrealdb&lt;/code&gt; repo.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The week between release and public sync gives us a private window to triage any issues or vulnerabilities reported against the new version before the corresponding commits become visible. We can ship a fix and roll it out across SurrealDB Cloud before the underlying issue is public. This is the same model used by most database and infrastructure projects at comparable scale, and it lets us close security issues as quickly as they're reported.&lt;/p&gt;

&lt;p&gt;A few practical notes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Issues, PRs, and discussions stay on the public repo&lt;/strong&gt;. Keep filing them on &lt;code&gt;surrealdb/surrealdb&lt;/code&gt;. We watch and respond to them daily.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Security reports&lt;/strong&gt;. Use the GitHub Security Advisory flow on the public repo, or email &lt;code&gt;security@surrealdb.com&lt;/code&gt;. Both reach us privately, and are the right route for anything you don't want disclosed publicly.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Community PRs&lt;/strong&gt;. We're still very keen for community contributions. They get reviewed against the public repo as before, and brought into the private tree as part of the normal cycle.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We don't expect this to be visible to most users in day-to-day use. If you follow commit-by-commit development on the public repo, the experience will change: you'll see batched updates around each release rather than a continuous stream. We think the security benefit is worth that tradeoff, and we plan to keep working this way for the foreseeable future.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;SurrealDB 3.1 is available now. Head to the &lt;a href="https://surrealdb.com/install" rel="noopener noreferrer"&gt;install page&lt;/a&gt; to grab it for your platform.&lt;/p&gt;

&lt;p&gt;If you're running on SurrealDB Cloud, instances are being rolled to 3.1 as part of our standard release cycle. No action needed.&lt;/p&gt;

&lt;p&gt;Read the &lt;a href="https://surrealdb.com/releases/3.1.0" rel="noopener noreferrer"&gt;full release notes&lt;/a&gt; for the complete list of changes, and come say hello on &lt;a href="https://discord.gg/surrealdb" rel="noopener noreferrer"&gt;Discord&lt;/a&gt; if you hit anything along the way.&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>database</category>
      <category>news</category>
      <category>graphql</category>
    </item>
    <item>
      <title>Kreuzberg &amp; SurrealDB: from unstructured documents to hybrid retrieval</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Tue, 12 May 2026 12:35:55 +0000</pubDate>
      <link>https://dev.to/surrealdb/kreuzberg-surrealdb-from-unstructured-documents-to-hybrid-retrieval-3657</link>
      <guid>https://dev.to/surrealdb/kreuzberg-surrealdb-from-unstructured-documents-to-hybrid-retrieval-3657</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://x.com/IgnacioPaz87" rel="noopener noreferrer"&gt;Ignacio Paz&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We’re excited to share a new partner integration: &lt;code&gt;kreuzberg-surrealdb&lt;/code&gt;, a connector that bridges the Kreuzberg document intelligence framework directly into SurrealDB. This integration was created by the Kreuzberg team and we are excited to have this functionality available now in SurrealDB.&lt;/p&gt;

&lt;p&gt;Kreuzberg extracts, chunks, and generates embeddings from 88+ document formats, while SurrealDB provides a multi-model database for AI applications, combining documents, graphs, vectors, and full-text search in a single system.&lt;/p&gt;

&lt;p&gt;Together, they make it easy to build document search and RAG pipelines.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the integration does
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0mi2ox6g7h6vmbu2d7x3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0mi2ox6g7h6vmbu2d7x3.png" alt="document extraction" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;kreuzberg-surrealdb&lt;/code&gt; handles the full ingestion workflow:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Automatic schema setup&lt;/li&gt;
&lt;li&gt;Content deduplication using SHA-256 hashing&lt;/li&gt;
&lt;li&gt;Storage and indexing in SurrealDB&lt;/li&gt;
&lt;li&gt;Documents ready for search immediately after ingest&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The integration supports two modes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;DocumentConnector:indexes full documents for BM25 keyword search.&lt;/li&gt;
&lt;li&gt;DocumentPipeline:chunks documents, generates embeddings, and enables semantic and hybrid search using HNSW vector indexes and Reciprocal Rank Fusion.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why it matters
&lt;/h2&gt;

&lt;p&gt;Building document search systems often requires combining multiple tools for extraction, chunking, embeddings, and storage.&lt;/p&gt;

&lt;p&gt;With &lt;code&gt;kreuzberg-surrealdb&lt;/code&gt;, the entire workflow runs through a &lt;strong&gt;single integration&lt;/strong&gt;—no schema boilerplate, no duplicate ingestion, and built-in support for &lt;strong&gt;keyword&lt;/strong&gt;, &lt;strong&gt;semantic&lt;/strong&gt;, and &lt;strong&gt;hybrid search&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Get started
&lt;/h2&gt;

&lt;p&gt;See how to get started in &lt;a href="https://surrealdb.com/docs/build/integrations/ai-frameworks/kreuzberg?utm_source=kreuzberg_blog&amp;amp;utm_medium=blog&amp;amp;utm_campaign=kreuzberg_blog" rel="noopener noreferrer"&gt;SurrealDB Docs: Kreuzberg Integration&lt;/a&gt;, and check out our example of &lt;a href="https://surrealdb.com/blog/how-to-build-a-knowledge-graph-for-ai?utm_source=kreuzberg_blog&amp;amp;utm_medium=blog&amp;amp;utm_campaign=kreuzberg_blog#parsing-unstructured-data" rel="noopener noreferrer"&gt;How to build a knowledge graph for AI with SurrealDB and Kreuzberg&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>kreuzberg</category>
      <category>database</category>
      <category>integration</category>
    </item>
    <item>
      <title>Schema migrations in SurrealDB: A local dev workflow</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Fri, 08 May 2026 12:42:16 +0000</pubDate>
      <link>https://dev.to/surrealdb/schema-migrations-in-surrealdb-a-local-dev-workflow-1ijm</link>
      <guid>https://dev.to/surrealdb/schema-migrations-in-surrealdb-a-local-dev-workflow-1ijm</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://github.com/itsezc" rel="noopener noreferrer"&gt;Chiru Boggavarapu&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This post walks through a proper migration workflow for local development using &lt;strong&gt;SurrealKit&lt;/strong&gt;, an official tool from the SurrealDB team that handles schema sync, rollouts, seeding, and testing.&lt;/p&gt;

&lt;p&gt;If you've spent any time working with SurrealDB, you know that with great flexibility comes the responsibility of managing your schema carefully. You're iterating quickly, adding tables, refining field definitions, and removing what you no longer need, and before long your local database can drift out of sync with what your code expects.&lt;/p&gt;




&lt;h2&gt;
  
  
  What is SurrealKit?
&lt;/h2&gt;

&lt;p&gt;SurrealKit is a CLI tool that manages your SurrealDB schema through &lt;code&gt;.surql&lt;/code&gt; files. You define your schema as files, and SurrealKit keeps your database in sync with them.&lt;/p&gt;

&lt;p&gt;It has two main modes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Sync&lt;/strong&gt;: a fast, declarative approach for local and dev environments. Your files are the source of truth. Add something and it gets created, change it and it gets updated, remove it and it gets deleted.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Rollout&lt;/strong&gt;: a more controlled migration path for shared or production databases, with planning, staged execution, and rollback support.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For local dev, you'll mostly live in &lt;code&gt;sync&lt;/code&gt;. Rollouts come into play when you're pushing to shared environments.&lt;/p&gt;




&lt;h2&gt;
  
  
  Getting Started
&lt;/h2&gt;

&lt;p&gt;Cargo is the Rust package manager. It downloads a Rust package's dependencies, compiles the packages, and makes distributable packages.&lt;/p&gt;

&lt;p&gt;Install via Cargo&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cargo &lt;span class="nb"&gt;install &lt;/span&gt;surrealkit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or grab a prebuilt binary from the GitHub releases page if you don't want to compile it.&lt;/p&gt;

&lt;p&gt;Initialise a project&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit init
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This creates a &lt;code&gt;/database&lt;/code&gt; directory with the scaffolding you need. SurrealKit connects to your database using environment variables, so add these to your &lt;code&gt;.env&lt;/code&gt;:&lt;/p&gt;

&lt;p&gt;.env&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="s"&gt;SURREALDB_HOST=localhost:8000&lt;/span&gt;
&lt;span class="s"&gt;SURREALDB_NAME=myapp&lt;/span&gt;
&lt;span class="s"&gt;SURREALDB_NAMESPACE=development&lt;/span&gt;
&lt;span class="s"&gt;SURREALDB_USER=root&lt;/span&gt;
&lt;span class="s"&gt;SURREALDB_PASSWORD=secret&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It supports both &lt;code&gt;DATABASE_HOST&lt;/code&gt; and &lt;code&gt;PUBLIC_DATABASE_HOST&lt;/code&gt; variants, which is handy if you're working in a SvelteKit or similar setup where env vars are split by visibility.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Local Dev Workflow
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Writing Your Schema
&lt;/h3&gt;

&lt;p&gt;Schema files live in &lt;code&gt;database/schema/&lt;/code&gt;. Each file is a &lt;code&gt;.surql&lt;/code&gt; file with your table definitions, indexes, access rules — whatever you'd normally write in SurrealQL.&lt;/p&gt;

&lt;p&gt;The structure is entirely up to you. You might have one file per table, or group related things together. Either way, SurrealKit tracks what's in those files and reconciles it with what's actually in your database.&lt;/p&gt;

&lt;h3&gt;
  
  
  Syncing Changes
&lt;/h3&gt;

&lt;p&gt;When you've made a change and want to apply it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit &lt;span class="nb"&gt;sync&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's it. SurrealKit diffs your schema files against the current database state and applies what's changed. If you deleted a table definition from your files, it removes it from the database too. Everything stays in sync.&lt;/p&gt;

&lt;p&gt;For active development sessions, watch mode is where you'll spend most of your time:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit &lt;span class="nb"&gt;sync&lt;/span&gt; &lt;span class="nt"&gt;--watch&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This watches your &lt;code&gt;database/schema/&lt;/code&gt; directory and resyncs automatically whenever you save a file. It handles deletions too — if you remove a definition, it gets cleaned up. No manual intervention needed.&lt;/p&gt;




&lt;h2&gt;
  
  
  Moving to Shared Environments
&lt;/h2&gt;

&lt;p&gt;The sync approach is great for local databases you own completely. But the moment you're working against a shared or staging database, you need more control. That's what rollouts are for.&lt;/p&gt;

&lt;h3&gt;
  
  
  Baseline First
&lt;/h3&gt;

&lt;p&gt;Before your first rollout on an existing database:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit rollout baseline
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This records the current state so SurrealKit knows what it's working from.&lt;/p&gt;

&lt;h3&gt;
  
  
  Plan → Start → Complete
&lt;/h3&gt;

&lt;p&gt;When you're ready to ship a schema change:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit rollout plan &lt;span class="nt"&gt;--name&lt;/span&gt; add_user_indexes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This generates a TOML manifest in &lt;code&gt;database/rollouts/&lt;/code&gt; describing exactly what will change. You can review it, commit it, get it reviewed. Treat it like a pull request for your schema.&lt;/p&gt;

&lt;p&gt;When you're ready to apply, you'll reference the manifest by its full filename. SurrealKit names these with a timestamp prefix: &lt;code&gt;20260302153045&lt;/code&gt; translates to 2nd March 2026 at 15:30:45. This is the time the plan was generated, and it's there so rollouts always have a guaranteed sort order. If two people generate a plan on the same day, the timestamps keep them distinct and sequential. You'll see the full name in your &lt;code&gt;database/rollouts/&lt;/code&gt; directory after running &lt;code&gt;plan&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;When you're ready to apply:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit rollout start 20260302153045__add_user_indexes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This runs the non-destructive part of the migration. Once your application is deployed and you're confident everything is working:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit rollout &lt;span class="nb"&gt;complete &lt;/span&gt;20260302153045__add_user_indexes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This finishes the migration, including any cleanup of legacy objects.&lt;/p&gt;

&lt;p&gt;If something goes wrong mid-rollout:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit rollout rollback 20260302153045__add_user_indexes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You can also validate a manifest without touching the database:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit rollout lint 20260302153045__add_user_indexes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And check what state a rollout is in:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit rollout status
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;SurrealKit tracks rollout state in the database itself, so it's always resumable.&lt;/p&gt;




&lt;h2&gt;
  
  
  Seeding
&lt;/h2&gt;

&lt;p&gt;SurrealKit has a built-in seeding system for populating your local database with test data:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit seed
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Seed files live alongside your schema in the &lt;code&gt;/database&lt;/code&gt; directory. Useful for onboarding new team members quickly — they clone the repo, init SurrealKit, run sync and seed, and they're good to go.&lt;/p&gt;




&lt;h2&gt;
  
  
  Testing
&lt;/h2&gt;

&lt;p&gt;One of the more interesting features is the declarative testing framework. You write TOML test suites in &lt;code&gt;database/tests/suites/&lt;/code&gt; and run them with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit &lt;span class="nb"&gt;test&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each suite runs in an isolated ephemeral namespace, so tests can't interfere with each other or your actual data. You can test SQL assertions, permission rules, schema metadata, and even HTTP API endpoints.&lt;/p&gt;

&lt;p&gt;A simple example, checking that a guest can't create an order:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"security_smoke"&lt;/span&gt;

&lt;span class="nn"&gt;[[cases]]&lt;/span&gt;
&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"guest_cannot_create_order"&lt;/span&gt;
&lt;span class="py"&gt;kind&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"sql_expect"&lt;/span&gt;
&lt;span class="py"&gt;actor&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"guest"&lt;/span&gt;
&lt;span class="py"&gt;sql&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"CREATE order CONTENT { total: 10 };"&lt;/span&gt;
&lt;span class="py"&gt;allow&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;
&lt;span class="py"&gt;error_contains&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"permission"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You can also test permission matrices across a whole table at once, which is handy for verifying your access rules haven't regressed after a schema change.&lt;/p&gt;

&lt;p&gt;For CI, add &lt;code&gt;--json-out&lt;/code&gt; to get machine-readable output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;surrealkit &lt;span class="nb"&gt;test&lt;/span&gt; &lt;span class="nt"&gt;--json-out&lt;/span&gt; database/tests/report.json
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The command exits non-zero on any failure, so it integrates cleanly with GitHub Actions and other CI/CD systems seamlessly.&lt;/p&gt;




&lt;h2&gt;
  
  
  Putting It Together
&lt;/h2&gt;

&lt;p&gt;Here's what a typical day looks like with this setup:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Pull the latest changes&lt;/li&gt;
&lt;li&gt;Run &lt;code&gt;surrealkit sync&lt;/code&gt; to update your local DB&lt;/li&gt;
&lt;li&gt;Edit schema files in &lt;code&gt;database/schema/&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Use &lt;code&gt;surrealkit sync --watch&lt;/code&gt; while you work&lt;/li&gt;
&lt;li&gt;When you're done, run &lt;code&gt;surrealkit test&lt;/code&gt; to make sure nothing broke&lt;/li&gt;
&lt;li&gt;Commit your schema files alongside your code&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For production changes, swap step 4 for &lt;code&gt;surrealkit rollout plan&lt;/code&gt; and follow the plan → start → complete flow.&lt;/p&gt;

&lt;p&gt;Would love to hear how you get on with SurrealKit. &lt;a href="https://discord.com/invite/surrealdb" rel="noopener noreferrer"&gt;Jump into Discord&lt;/a&gt; and let us know how it goes or if you need a hand along the way.&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>surrealkit</category>
      <category>database</category>
      <category>schema</category>
    </item>
    <item>
      <title>New SurrealDB docs search using hybrid search and HNSW/BM25 reranking</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Fri, 01 May 2026 15:32:31 +0000</pubDate>
      <link>https://dev.to/surrealdb/new-surrealdb-docs-search-using-hybrid-search-and-hnswbm25-reranking-51jd</link>
      <guid>https://dev.to/surrealdb/new-surrealdb-docs-search-using-hybrid-search-and-hnswbm25-reranking-51jd</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://x.com/mithridates" rel="noopener noreferrer"&gt;Dave MacLeod&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Trying to produce relevant search results is one of the hardest things to get right when building an app. Searching can be done in a lot of ways: basic text search, full-text search, vector search, and often even graph and geospatial.&lt;/p&gt;

&lt;p&gt;This post introduces an example that includes two of these models: full-text and vector search together. These two tend to involve quite a bit of SurrealQL code, because:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;To use full-text search, you first have to define how you want text to be split up and modified. Do you want it to be case-sensitive? Split by whitespace, or some other way? Modified to root forms based on the language of the text? (And so on...)&lt;/li&gt;
&lt;li&gt;To use vector search, you have to generate embeddings for pieces of text and know how to query them.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What makes this example special is that it is the search function you use every time you type some text into the "Search the docs" part of the SurrealDB documentation! This functionality was recently implemented, and because it's open source we are able to show you the ins and outs of how it was done.&lt;/p&gt;




&lt;h2&gt;
  
  
  How Doc Search Is Implemented
&lt;/h2&gt;

&lt;p&gt;The &lt;a href="https://github.com/surrealdb/docs.surrealdb.com/blob/main/search/README.md" rel="noopener noreferrer"&gt;README&lt;/a&gt; in the &lt;a href="https://github.com/surrealdb/docs.surrealdb.com/tree/main/search" rel="noopener noreferrer"&gt;/search&lt;/a&gt; section of the docs repo goes into a good amount of detail. &lt;/p&gt;

&lt;p&gt;In addition, you can see the mechanics of the search itself inside  this &lt;a href="https://github.com/surrealdb/docs.surrealdb.com/blob/main/src/utils/search.ts" rel="noopener noreferrer"&gt;search.ts file&lt;/a&gt; in another section.&lt;/p&gt;

&lt;p&gt;The entire implementation is done using SurrealDB, along with Bun to deploy the website and an OpenAI API key. You can go to &lt;a href="https://github.com/surrealdb/docs.surrealdb.com/tree/main/search#local-development" rel="noopener noreferrer"&gt;this part of the page&lt;/a&gt; to deploy it locally and test it out. &lt;/p&gt;

&lt;p&gt;Here is what the final result looks like:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fgsy92u2q866ijtmk97w5.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fgsy92u2q866ijtmk97w5.png" alt="How Doc Search Is Implemented" width="800" height="679"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Since the README file already goes into detail about how to do the deployment yourself, in this post we will instead take a closer look at the SurrealQL statements themselves to see how they work and how they might help you do something similar in your own tools and apps.&lt;/p&gt;




&lt;h2&gt;
  
  
  The role SurrealDB Plays
&lt;/h2&gt;

&lt;p&gt;As the README mentions, the search functionality uses hybrid search:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Hybrid search for the SurrealDB documentation. Combines BM25 full-text search with OpenAI vector embeddings, fused via Reciprocal Rank Fusion (RRF) inside SurrealDB.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://surrealdb.com/docs/learn/data-models/vector-search/hybrid-search" rel="noopener noreferrer"&gt;Hybrid search&lt;/a&gt; allows you to combine full-text search with vector search, giving you the best of both worlds.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Vector search&lt;/strong&gt; is useful because it is capable of going beyond the outward appearance of words to get to their semantic meaning. For example, the term lead in the following paragraph is used to mean two different things. We can recognize that as humans, and LLM models are also able to do the same, returning the output as an embedding (a large array of numbers representing semantic space).&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The project &lt;strong&gt;lead&lt;/strong&gt; frowned and took a hard look at the results. They were clear as day. There was far too much lead content in the water. That's why the town had gotten sick!&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;However, full-text search is useful too because it excels at slicing and modifying text into tokens. To show what that looks like, here's what the full-text analyzer looks like that is used in the docs search.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt;
    &lt;span class="n"&gt;TOKENIZERS&lt;/span&gt; &lt;span class="n"&gt;blank&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;class&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;camel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;punct&lt;/span&gt;
    &lt;span class="n"&gt;FILTERS&lt;/span&gt; &lt;span class="n"&gt;snowball&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;english&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The easiest way to understand what all the tokenizers and filters do in an analyzer is by putting a string into the &lt;a href="https://surrealdb.com/docs/reference/query-language/functions/database-functions/search#searchanalyze" rel="noopener noreferrer"&gt;search::analyze()&lt;/a&gt; function, which returns an array. To avoid showing a huge array in this blog post, we'll call &lt;code&gt;.join(' ')&lt;/code&gt; on the output to join them back into a single string.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="k"&gt;analyze&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nv"&gt;"simple"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; 
    &lt;span class="nv"&gt;"The project lead frowned and took a hard look at the results. They were clear as day. There was far too much lead content in the water. That's how the town had gotten sick!"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;' '&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;As the output shows, using this analyzer will allow you to match on terms like "FroWNinG" which will turn into "frown" before being compared against a search string.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"the project lead frown and took a hard look at the result . they were clear as day . there was far too much lead content in the water . that ' s how the town had gotten sick !"&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;So if vector search and full-text search each have their advantages, how can you combine the two? It would be nice if you could somehow plug them into a function that does this.&lt;/p&gt;

&lt;p&gt;And it turns out that SurrealDB has exactly that! It is called &lt;a href="https://surrealdb.com/docs/reference/query-language/functions/database-functions/search#searchrrf" rel="noopener noreferrer"&gt;search::rrf()&lt;/a&gt; and was added during the 3.0 beta period. Here is what it looks like in practice.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- ── Reciprocal Rank Fusion ──&lt;/span&gt;
&lt;span class="c1"&gt;-- Combines the four ranked lists into a single ranking.&lt;/span&gt;
&lt;span class="c1"&gt;-- RRF scores each result as: sum(1 / (k + rank_in_list))&lt;/span&gt;
&lt;span class="c1"&gt;-- across all lists the result appears in.&lt;/span&gt;
&lt;span class="c1"&gt;--   arg 1: array of ranked lists to fuse&lt;/span&gt;
&lt;span class="c1"&gt;--   arg 2: k=60 (smoothing constant, standard RRF default)&lt;/span&gt;
&lt;span class="c1"&gt;--   arg 3: limit=80 (max candidates to consider)&lt;/span&gt;
&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;fused&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;rrf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;page_ft&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;page_vs&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;section_ft&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;section_vs&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; 
    &lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; 
    &lt;span class="mi"&gt;80&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;As you can probably guess, it is fusing the results of previous queries for full-text search on pages and sections (&lt;code&gt;$page_ft, $section_ft&lt;/code&gt;) together with vector searches on pages and sections (&lt;code&gt;$page_vs, $section_vs&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;Full-text searches are done using the &lt;code&gt;@@&lt;/code&gt; operator (the "matches" operator). This operator works as long as you have a index on a field that uses a &lt;code&gt;FULLTEXT ANALYZER&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;page_ft_title&lt;/span&gt;       &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;title&lt;/span&gt;       &lt;span class="n"&gt;FULLTEXT&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt; &lt;span class="n"&gt;BM25&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;page_ft_breadcrumb&lt;/span&gt;  &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;breadcrumb&lt;/span&gt;  &lt;span class="n"&gt;FULLTEXT&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt; &lt;span class="n"&gt;BM25&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;page_ft_description&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;description&lt;/span&gt; &lt;span class="n"&gt;FULLTEXT&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt; &lt;span class="n"&gt;BM25&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;page_ft_content&lt;/span&gt;     &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt;     &lt;span class="n"&gt;FULLTEXT&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt; &lt;span class="n"&gt;BM25&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;page_ft_path&lt;/span&gt;        &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;path&lt;/span&gt;        &lt;span class="n"&gt;FULLTEXT&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt; &lt;span class="n"&gt;BM25&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In between these two &lt;code&gt;@&lt;/code&gt; operators you can insert a number to access the &lt;code&gt;search::score()&lt;/code&gt; for each field. This will make sense once you take a look at these two queries. Each of them is querying multiple fields, each of which will return a score thanks to &lt;code&gt;BM25&lt;/code&gt; defined in the field. To know which field is which, you put a reference number in between the &lt;code&gt;@&lt;/code&gt; operators like &lt;code&gt;@0@&lt;/code&gt; and &lt;code&gt;@2@&lt;/code&gt;. They can then be added together or multiplied or whatever logic you would like to use.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;page_ft&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
        &lt;span class="k"&gt;SELECT&lt;/span&gt;
            &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nv"&gt;"page"&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;page_path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;collection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;breadcrumb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;25&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;ft_score&lt;/span&gt;
        &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;
        &lt;span class="k"&gt;WHERE&lt;/span&gt;
            &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
            &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="n"&gt;title&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
            &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="n"&gt;breadcrumb&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
            &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="n"&gt;description&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
            &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
        &lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;ft_score&lt;/span&gt; &lt;span class="k"&gt;DESC&lt;/span&gt;
        &lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;section_ft&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; 
        &lt;span class="k"&gt;SELECT&lt;/span&gt;
            &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nv"&gt;"section"&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;string&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;"#"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;anchor&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;page_path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;collection&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;collection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;breadcrumb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;25&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;ft_score&lt;/span&gt;
        &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;section&lt;/span&gt;
        &lt;span class="k"&gt;WHERE&lt;/span&gt;
            &lt;span class="n"&gt;title&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
            &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="n"&gt;breadcrumb&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
            &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;
        &lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;ft_score&lt;/span&gt; &lt;span class="k"&gt;DESC&lt;/span&gt;
        &lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The vector search query also uses its own index. Note that you don't specifically need an index to use vector search, as you can go with a &lt;a href="https://surrealdb.com/docs/reference/query-language/functions/database-functions/vector#vectorsimilaritycosine" rel="noopener noreferrer"&gt;brute force function&lt;/a&gt; such as &lt;code&gt;vector::similarity::cosine()&lt;/code&gt; instead. However, an &lt;code&gt;HNSW&lt;/code&gt; index instead can speed up the process dramatically if you don't mind sacrificing a bit of precision. Here the index is defined with a dimension of 1536 to match the length of the array returned by OpenAI's text-embedding-3-small model.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- HNSW index for approximate nearest-neighbour vector search.&lt;/span&gt;
&lt;span class="c1"&gt;-- Dimension 1536 matches OpenAI text-embedding-3-small output.&lt;/span&gt;
&lt;span class="c1"&gt;-- Cosine distance is the standard metric for normalised text embeddings.&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;page_embedding_hnsw&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt;
    &lt;span class="n"&gt;HNSW&lt;/span&gt; &lt;span class="n"&gt;DIMENSION&lt;/span&gt; &lt;span class="mi"&gt;1536&lt;/span&gt; &lt;span class="n"&gt;DIST&lt;/span&gt; &lt;span class="n"&gt;COSINE&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Having an &lt;code&gt;HNSW&lt;/code&gt; index also lets you use the &lt;code&gt;&amp;lt;||&amp;gt;&lt;/code&gt; operator (the KNN, or K-nearest neighbour) operator when performing the query. The comments above the code explain what each of the two numbers mean. Here, the &lt;code&gt;|30,100|&lt;/code&gt; syntax means to return 30 neighbours among up to 100 candidates.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- ── Page vector search ──&lt;/span&gt;
&lt;span class="c1"&gt;-- Finds the 30 pages whose embeddings are closest to the&lt;/span&gt;
&lt;span class="c1"&gt;-- query embedding. The &amp;lt;|30,100|&amp;gt; syntax means: return 30&lt;/span&gt;
&lt;span class="c1"&gt;-- neighbours, exploring up to 100 candidates in the HNSW&lt;/span&gt;
&lt;span class="c1"&gt;-- graph (higher = more accurate but slower).&lt;/span&gt;
&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;page_vs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; 
    &lt;span class="k"&gt;SELECT&lt;/span&gt;
        &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nv"&gt;"page"&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;page_path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;collection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;breadcrumb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;vector&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;distance&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;knn&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;distance&lt;/span&gt;
    &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;
    &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;|&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="o"&gt;|&amp;gt;&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;qvec&lt;/span&gt;
    &lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;distance&lt;/span&gt; &lt;span class="k"&gt;ASC&lt;/span&gt;
    &lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- ── Section vector search ──&lt;/span&gt;
&lt;span class="c1"&gt;-- Same as page vector search but for H2 sections.&lt;/span&gt;
&lt;span class="c1"&gt;-- Pulls the parent page's path and collection via the&lt;/span&gt;
&lt;span class="c1"&gt;-- record link (page.path, page.collection).&lt;/span&gt;
&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;section_vs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
    &lt;span class="k"&gt;SELECT&lt;/span&gt;
        &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nv"&gt;"section"&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;string&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;"#"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;anchor&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;page_path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;collection&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;collection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;breadcrumb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;vector&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;distance&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;knn&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;distance&lt;/span&gt;
    &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;section&lt;/span&gt;
    &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;|&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="o"&gt;|&amp;gt;&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;qvec&lt;/span&gt;
    &lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;distance&lt;/span&gt; &lt;span class="k"&gt;ASC&lt;/span&gt;
    &lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So that is how the magic happens. Well, part of it. Be sure to check out the rest of the code which shows what happens after the SurrealDB query returns. For example, as the database is unaware of which collections it is searching through, some modifications are made on the SDK side to give SDK docs less precedence compared to more core docs. This is something that you could perhaps do on the database side if you wanted (starting with &lt;code&gt;DEFINE TABLE sdk_page&lt;/code&gt; instead of &lt;code&gt;DEFINE TABLE page&lt;/code&gt; for example) but that will always depend on your own case.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// ──────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span class="c1"&gt;// Post-retrieval boosting&lt;/span&gt;
&lt;span class="c1"&gt;//&lt;/span&gt;
&lt;span class="c1"&gt;// After RRF fusion, we apply multiplicative boosts to adjust&lt;/span&gt;
&lt;span class="c1"&gt;// rankings based on signals the database query can't capture:&lt;/span&gt;
&lt;span class="c1"&gt;// title similarity, content type (page vs section), source&lt;/span&gt;
&lt;span class="c1"&gt;// collection, and comparison-query detection.&lt;/span&gt;
&lt;span class="c1"&gt;// ──────────────────────────────────────────────────────────&lt;/span&gt;

&lt;span class="cm"&gt;/**
 * Non-SDK doc collections get a small ranking boost because
 * generic queries like "authentication" should prefer the
 * core concept page over an SDK API reference page that
 * happens to mention auth as one of many methods.
 */&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;CORE_COLLECTIONS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-surrealdb&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-surrealql&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-tutorials&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-cloud&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-surrealist&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-surrealml&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-surrealkv&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;doc-integrations&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Simplifying the Example
&lt;/h2&gt;

&lt;p&gt;Since you might not have the head space at the moment to play around with the existing code demonstrated in this blog post, let's finish up with the same pattern as above in a very simplified form. This will let you copy and paste the example into the online &lt;a href="https://app.surrealdb.com/?_gl=1*n737cr*_gcl_aw*R0NMLjE3NzQ2MjU1MDcuQ2owS0NRancxWmpPQmhDbUFSSXNBRER1RlRDVV9VN2JENUdhMFQzTm5mV0paMzNOVkxiTDNROVFOSTlUckhRUHV0V2FVN1l2czJ0MlNGQWFBc3hCRUFMd193Y0I.*FPAU*NTM0Mzg1Mzg1LjE3NzA2NTk1MTg.*_ga*NDAyMzA0NTY4LjE3NzEyNDU1Nzg.*_ga_J1NWM32T1V*czE3Nzc2NDUxMDkkbzQxMyRnMSR0MTc3NzY0NTE5NCRqNjAkbDAkaDQwOTUwMzMzMA..*_fplc*OVhscHhYNEtSdG1MUlk4bUJqSnJVaTVXSDBNSTEyUG1DMFUlMkY3VXoycHhQbFVUaHFGY1dvR2JUdHFuVkhXWEY0JTJGTkYlMkZnUDVVajNoalZ3V095cG93cGd2N0c2ZzdMZk5hM1hLS2xxMXYyNG42R3hUZ0UlMkY0SXdsUnFWak9DWkElM0QlM0Q." rel="noopener noreferrer"&gt;Surrealist UI&lt;/a&gt; to give it a try yourself.&lt;/p&gt;

&lt;p&gt;The content below holds all the same features: a full-text analyzer, a full-text and vector index, a full-text and vector query, and a final query that combines the two into hybrid search results. However, it has all been simplified to a single field for text content and embeddings, and a single document table that is being searched on. In addition, the embeddings have been cut down to an array of just ten numbers in length. An array that small is very imprecise, but just long enough to serve for our simple example.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Define an analyzer&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt; &lt;span class="n"&gt;TOKENIZERS&lt;/span&gt; &lt;span class="n"&gt;blank&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;class&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;camel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;punct&lt;/span&gt; &lt;span class="n"&gt;FILTERS&lt;/span&gt; &lt;span class="n"&gt;SNOWBALL&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;en&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="c1"&gt;-- Attach it to an index&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;ft&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="n"&gt;FULLTEXT&lt;/span&gt; &lt;span class="n"&gt;ANALYZER&lt;/span&gt; &lt;span class="k"&gt;simple&lt;/span&gt; &lt;span class="n"&gt;BM25&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;75&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="c1"&gt;-- Create a vector index too&lt;/span&gt;
&lt;span class="n"&gt;DEFINE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;OVERWRITE&lt;/span&gt; &lt;span class="n"&gt;page_embedding_hnsw&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt; &lt;span class="n"&gt;FIELDS&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="n"&gt;HNSW&lt;/span&gt; &lt;span class="n"&gt;DIMENSION&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt; &lt;span class="n"&gt;DIST&lt;/span&gt; &lt;span class="n"&gt;COSINE&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- Add some sample data&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;one&lt;/span&gt; &lt;span class="k"&gt;SET&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;   &lt;span class="nv"&gt;"Elves and halflings and witches and such"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0019&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0142&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0080&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0664&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0173&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0109&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0066&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0045&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0204&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0087&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;
    &lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;two&lt;/span&gt; &lt;span class="k"&gt;SET&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;   &lt;span class="nv"&gt;"Is that a wizard?"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                        &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0010&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0051&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0207&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0787&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0061&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0127&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0172&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0097&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0077&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0120&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;
    &lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;three&lt;/span&gt; &lt;span class="k"&gt;SET&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;"Databases are used to store information"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0317&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0112&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0118&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0348&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0061&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0114&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0310&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0117&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0034&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0195&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;
    &lt;span class="k"&gt;NONE&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="c1"&gt;-- A sample query and its embedding&lt;/span&gt;
&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;witch_text&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;"witches"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;witch_embed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0059&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0081&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0475&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0020&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0295&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0183&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0170&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0048&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0286&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;

&lt;span class="c1"&gt;-- Get the full-text score&lt;/span&gt;
&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;fts_score&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
        &lt;span class="k"&gt;SELECT&lt;/span&gt;
            &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;ft_score&lt;/span&gt;
        &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;
        &lt;span class="k"&gt;WHERE&lt;/span&gt;
            &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;witch_text&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- Get the vector score&lt;/span&gt;
&lt;span class="n"&gt;LET&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;vector_score&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; 
    &lt;span class="k"&gt;SELECT&lt;/span&gt;
        &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;vector&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;distance&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;knn&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;distance&lt;/span&gt;
    &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;
    &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;|&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="o"&gt;|&amp;gt;&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;witch_embed&lt;/span&gt;
    &lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;distance&lt;/span&gt; &lt;span class="k"&gt;ASC&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- Combine the results as a hybrid score&lt;/span&gt;
&lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;rrf&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;fts_score&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;vector_score&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;80&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We can see in the output that while only one document had a match on the word "witches" (and thus a value for ft_score), the vector search portion has been able to conclude that the sentence about wizards is semantically a bit closer than the one about databases. This would be a lot more accurate of course if the embedding were longer than 10 elements, but even at this length we can still see its potential.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Elves and halflings and witches and such"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"distance"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.2789615948854415&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"ft_score"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.4782197177410126&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"document:one"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"rrf_score"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.024691358024691357&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Is that a wizard?"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"distance"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.385041442417419&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"document:two"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"rrf_score"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.012195121951219513&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Databases are used to store information"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"distance"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.3876749269204385&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"document:three"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"rrf_score"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.012048192771084338&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Hopefully the simplified example has made it easy to follow the logic we use in our own doc search and has given you some ideas for how to implement patterns yourself. Have any questions or comments about hybrid search? Feel free to &lt;a href="https://surrealdb.com/docs/build/deployment/surrealdb-cloud/getting-started/create-an-instance" rel="noopener noreferrer"&gt;Create a free SurrealDB Cloud instance today to get started&lt;/a&gt;, and drop by &lt;a href="https://discord.com/invite/surrealdb" rel="noopener noreferrer"&gt;our Discord community&lt;/a&gt; to discuss anything on your mind with the SurrealDB community and staff.&lt;/p&gt;

</description>
      <category>surrealdb</category>
      <category>database</category>
      <category>devrel</category>
      <category>hybridsearch</category>
    </item>
    <item>
      <title>How to get near-perfect, deterministic accuracy from your AI agents</title>
      <dc:creator>Mark Gyles</dc:creator>
      <pubDate>Thu, 23 Apr 2026 22:23:21 +0000</pubDate>
      <link>https://dev.to/surrealdb/how-to-get-near-perfect-deterministic-accuracy-from-your-ai-agents-56ma</link>
      <guid>https://dev.to/surrealdb/how-to-get-near-perfect-deterministic-accuracy-from-your-ai-agents-56ma</guid>
      <description>&lt;p&gt;Author: &lt;a href="https://www.linkedin.com/in/mpenaroza/" rel="noopener noreferrer"&gt;Matthew Penaroza&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I have spent a lot of time working on large-scale agent architectures with some of the largest organizations in the world, and the single most common mistake I see teams make is assuming their accuracy problems are model problems. They almost never are. Your agent is typically not struggling because the LLM is weak. It is struggling because your retrieval layer is feeding it bad context. Fix your retrieval, and accuracy jumps. Make your retrieval learn from its own results, and accuracy becomes deterministic. Here is what that looks like in practice.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt; - Agent accuracy problems are almost always retrieval problems, not model problems. At scale, vector search alone is not enough. You need structured filters, graph traversal, temporal constraints, and vector similarity composed in a single atomic query to get to ~90–95% accuracy. To close the remaining gap to ~99%+, you need reasoning graphs and retrieval graphs: structured graph edges that capture the agent's logic and retrieval patterns, feeding back into the pipeline so the system learns which approaches produce correct outcomes. Both require a multi-model database with ACID transactions across every data model, which is exactly what we designed SurrealDB to be.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  The problem: why agent accuracy hits a ceiling
&lt;/h2&gt;

&lt;p&gt;Every agent follows the same loop: retrieve context, reason over it, act. The quality of the action is bounded by the quality of the context. I have seen perfectly capable models produce terrible outputs because the context they were given was noisy, and I have seen mid-tier models produce excellent outputs because the context was precise. The model matters less than most teams think.&lt;/p&gt;

&lt;p&gt;At small scale, none of this is a problem. If you have a few thousand documents, vector search returns good results, everything fits in a single database, and accuracy looks great. The problems emerge when data grows into the millions or billions of records. Suddenly more candidates look semantically similar, top-k selection becomes unstable, unrelated chunks bleed into the context window, and your agent starts hallucinating. The RAG survey literature (Gao et al., 2023) documents these failure modes extensively. Vector similarity is a useful ranking signal. It is not a control plane.&lt;/p&gt;

&lt;p&gt;What makes it worse is the typical production stack. Most teams are running Postgres for state, Neo4j for relationships, Pinecone for vectors, MongoDB for documents, sometimes Elasticsearch and a memory layer on top. Five or six independent systems, each with its own consistency model. When the graph database is 200ms behind the relational store, or the vector index has not ingested a document that was just written, your agent is reasoning over a version of reality that does not exist. It does not know the data is stale. It just makes a worse decision. And in multi-step workflows, each bad retrieval compounds into the next.&lt;/p&gt;

&lt;p&gt;So the question worth asking is not "which model should we use?" It is: how do you guarantee that the context reaching the model is valid, complete, current, and precise, even at scale? In my experience, the answer has two layers. The first gets you to roughly 90–95%. The second closes the gap to 99%+.&lt;/p&gt;

&lt;h2&gt;
  
  
  The accuracy staircase
&lt;/h2&gt;

&lt;p&gt;Each layer compounds on the last. Skip any step and accuracy hits a ceiling.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fa5lmoyx1h1gwz8fg7zvz.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fa5lmoyx1h1gwz8fg7zvz.png" alt="The accuracy staircase" width="800" height="894"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Getting to ~90–95%: scope-first retrieval
&lt;/h2&gt;

&lt;p&gt;The idea is straightforward: instead of running similarity search against everything and cleaning up the results, you constrain the candidate set before ranking. Define what counts as a valid result first. Rank inside that scope second.&lt;/p&gt;

&lt;p&gt;Think about what a single agent retrieval step actually needs, and what goes wrong without each layer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Structured filters&lt;/strong&gt; narrow by type, permissions, and tenancy. Without them, I have watched a support agent retrieve internal engineering postmortems and serve them verbatim to a customer, because the postmortem was semantically similar to the customer's question. The vector search was technically correct. The result was a data leak.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Temporal constraints&lt;/strong&gt; ensure the data is current. Without them, an agent recommending products will happily surface items that were discontinued months ago, because the old product description is still the closest embedding match. The customer clicks through to a dead link. Trust erodes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Graph traversal&lt;/strong&gt; follows real relationships: this customer owns this product, which has this known issue, which maps to this knowledge base article. Without it, your agent retrieves articles about similar-sounding problems on entirely different products. The answer reads plausibly but is wrong in the specifics, which is the worst kind of hallucination because it is hard to catch.&lt;/p&gt;

&lt;p&gt;Only after all of that scoping should vector similarity and full-text search run, ranking the remaining candidates by relevance. By this point, the candidate pool is already valid. Similarity is just picking the best among the valid.&lt;/p&gt;

&lt;p&gt;The problem is you cannot compose these operations across a fragmented stack. In a multi-database architecture, each is an independent query against an independent system. The application layer stitches the results together and hopes the data was consistent across all of them. It never fully is. You need a database that handles all of this natively, in a single query, against a single consistent snapshot. That is why we built SurrealDB the way we did: structured filters, temporal constraints, graph traversal, vector similarity, full-text search, and business rule enforcement all compose in one SurrealQL transaction.&lt;/p&gt;

&lt;p&gt;Here is what that looks like. One query, every scoping layer, against one consistent snapshot:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
    &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;vector&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;distance&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;knn&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;vec_dist&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;ft_score&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;vector&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;distance&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;knn&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;65&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;35&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;blend_score&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="k"&gt;search&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;highlight&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'&amp;lt;em&amp;gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'&amp;lt;/em&amp;gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;content_snippet&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;knowledge_base&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'support'&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;tenant&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;tenant&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;agent_principal&lt;/span&gt; &lt;span class="n"&gt;INSIDE&lt;/span&gt; &lt;span class="n"&gt;allowed_principals&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;updated_at&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="k"&gt;IN&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;customer&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;owns&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;product&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;has_issue&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;knowledge_base&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;content_embedding&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;|&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;40&lt;/span&gt;&lt;span class="o"&gt;|&amp;gt;&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query_embedding&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;query_text&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;blend_score&lt;/span&gt; &lt;span class="k"&gt;DESC&lt;/span&gt;
&lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That single statement does what would take four or five round-trips across a fragmented stack: permission checks, temporal filtering, graph traversal through the customer's actual product relationships, and hybrid vector + full-text ranking. One query. One transaction. One consistent snapshot. The line that does the most work is the graph traversal: &lt;code&gt;$customer-&amp;gt;owns-&amp;gt;product-&amp;gt;has_issue-&amp;gt;knowledge_base&lt;/code&gt;. That is not a join. It is a native graph traversal across edges, and it means the agent only sees knowledge base articles connected to the specific products this customer actually owns. Everything else is excluded before similarity even runs.&lt;/p&gt;

&lt;h3&gt;
  
  
  Two reasons fragmented stacks kill accuracy
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvx7ib5z59fa66hlpf13l.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvx7ib5z59fa66hlpf13l.png" alt="Two reasons fragmented stacks kill accuracy" width="800" height="1019"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Microsoft's GraphRAG work (Edge et al., 2024) demonstrated the same principle at the retrieval level: combining graph structure with similarity search significantly outperforms vector-only approaches. The CoALA framework (Sumers et al., 2024) describes it architecturally: agent memory must be consistent and accessible within a single reasoning cycle. A multi-model database that handles all data models under one transaction boundary is the simplest way I know to actually do both.&lt;/p&gt;

&lt;p&gt;This gets you to roughly 90–95% accuracy on large corpora. The consistency gaps are gone. The retrieval noise is way down. For many use cases, this is enough.&lt;/p&gt;

&lt;p&gt;But for agents that need to operate autonomously, make consequential decisions, or run in production without human oversight, the last 5–10% is where the difference between "mostly works" and "deterministic" lives. Closing that gap requires a different kind of mechanism.&lt;/p&gt;

&lt;h2&gt;
  
  
  Getting to ~99%+: reasoning graphs and retrieval graphs
&lt;/h2&gt;

&lt;p&gt;Scope-first retrieval fixes the shape of the funnel. Reasoning and retrieval graphs fix the funnel itself. They are the mechanism that turns a static retrieval pipeline into one that learns from its own results, and in my experience they are the single biggest differentiator between agents that are "pretty good" and agents that are genuinely deterministic.&lt;/p&gt;

&lt;p&gt;Every time an agent runs a retrieval pipeline and makes a decision, two graph structures get written in the same transaction as the action: a &lt;strong&gt;reasoning graph&lt;/strong&gt; (the agent's decision and the evaluated edges for each piece of evidence it considered) and a &lt;strong&gt;retrieval graph&lt;/strong&gt; (the operational details of how context was retrieved).&lt;/p&gt;

&lt;p&gt;I want to be specific about what these are. They are not log entries. They are not analytics events. They are graph edges stored in the same database as the data they describe, connecting the agent, the action, the customer, and the evidence records the agent evaluated. Written atomically with the agent's action and fully traversable with the same query language you use for everything else. ACID-guaranteed edges in the same graph your agent operates on. This is exactly why SurrealDB is a graph-native multi-model database: the reasoning and retrieval graphs have to be part of the same graph as the documents, the relationships, the vector embeddings, and the agent state, all under one transaction boundary. If they are not, the causal link between "what was retrieved," "how the agent reasoned about it," and "what it decided" breaks.&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;reasoning graph&lt;/strong&gt; is a set of edges radiating from the agent. A &lt;strong&gt;decided&lt;/strong&gt; edge connects the agent to the action, carrying confidence and the eventual outcome. But the reasoning itself is not a field on that edge. Each step of the agent's logic is its own &lt;strong&gt;evaluated&lt;/strong&gt; edge connecting the agent directly to the evidence it considered: &lt;code&gt;agent-&amp;gt;evaluated-&amp;gt;kb:4821&lt;/code&gt; with a verdict (used or rejected), the reason, and how much it moved the agent's confidence. The reasoning is not a string or an array. It is a set of traversable edges connecting the agent to every piece of evidence it touched. The &lt;strong&gt;retrieval graph&lt;/strong&gt; is a &lt;code&gt;retrieved_via&lt;/code&gt; edge from the action back to the customer, carrying which scope filters were applied, the graph path traversed, the candidate count before and after scoping, whether the top result was relevant, and the retrieval latency.&lt;/p&gt;

&lt;h3&gt;
  
  
  Anatomy of the Reasoning and Retrieval Graphs
&lt;/h3&gt;

&lt;p&gt;Every reasoning step and every retrieval detail is a graph edge. Traversable, not buried in strings.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F9f8wtnejx29cff55d4q1.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F9f8wtnejx29cff55d4q1.png" alt="Anatomy of the Reasoning and Retrieval Graphs" width="800" height="742"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The reason the reasoning has to be a graph, not a field, is that the pipeline planner needs to traverse individual evaluation steps across thousands of runs. If &lt;code&gt;kb:2201&lt;/code&gt; gets rejected in 94% of firmware queries, the planner can only discover that by traversing evaluated edges, not by parsing strings in a log. In SurrealQL, this is what the actual transaction looks like. The action is a node. The reasoning and retrieval are edges, created with &lt;code&gt;RELATE&lt;/code&gt;, connecting the agent to each piece of evidence and the action back to the customer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;BEGIN&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="n"&gt;support_action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;escalate_7291&lt;/span&gt; &lt;span class="n"&gt;CONTENT&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'escalate_to_engineering'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;customer&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;customer&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;acme&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;ticket&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;ticket&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;9913&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;created_at&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="n"&gt;RELATE&lt;/span&gt; &lt;span class="n"&gt;agent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;support_tier2&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;decided&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;support_action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;escalate_7291&lt;/span&gt; &lt;span class="n"&gt;CONTENT&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;confidence&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;94&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;created_at&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="n"&gt;RELATE&lt;/span&gt; &lt;span class="n"&gt;agent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;support_tier2&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;evaluated&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;kb&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;4821&lt;/span&gt; &lt;span class="n"&gt;CONTENT&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;support_action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;escalate_7291&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;step&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;verdict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'used'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;reason&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'Identical symptoms on same SKU'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;confidence_delta&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="n"&gt;RELATE&lt;/span&gt; &lt;span class="n"&gt;agent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;support_tier2&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;evaluated&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;kb&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;2201&lt;/span&gt; &lt;span class="n"&gt;CONTENT&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;support_action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;escalate_7291&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;step&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;verdict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'rejected'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;reason&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'Wrong product line'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;confidence_delta&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="n"&gt;RELATE&lt;/span&gt; &lt;span class="n"&gt;agent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;support_tier2&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;evaluated&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;kb&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;5540&lt;/span&gt; &lt;span class="n"&gt;CONTENT&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;support_action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;escalate_7291&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;step&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;verdict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'rejected'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;reason&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'Issue resolved, no longer active'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;confidence_delta&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="n"&gt;RELATE&lt;/span&gt; &lt;span class="n"&gt;support_action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;escalate_7291&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;retrieved_via&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;customer&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;acme&lt;/span&gt; &lt;span class="n"&gt;CONTENT&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;scope_filters&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="k"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'support'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tenant&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'acme'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;max_age&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'30d'&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="n"&gt;graph_path&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'customer:acme-&amp;gt;owns-&amp;gt;product:x100-&amp;gt;has_issue-&amp;gt;kb'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;candidates_before&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2847331&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;candidates_after&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;top_result_relevant&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;latency_ms&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;created_at&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="k"&gt;COMMIT&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The decision logic is not buried in strings. Every reasoning step is an edge in the graph. The &lt;code&gt;decided&lt;/code&gt; edge connects the agent to the action. Each &lt;code&gt;evaluated&lt;/code&gt; edge connects the agent to a specific piece of evidence it considered, carrying the verdict, the reason, and the confidence delta. The &lt;code&gt;retrieved_via&lt;/code&gt; edge connects the action back to the customer with the retrieval metadata. The evaluated edges are not magic. Your agent framework already evaluates candidates and produces a decision. The &lt;code&gt;RELATE&lt;/code&gt; statements just persist what the agent already computed. The only design requirement is that your agent emits structured evaluations per candidate rather than a single monolithic response.&lt;/p&gt;

&lt;p&gt;That graph structure gives you something most teams have never had: you can replay any agent run, traverse the exact reasoning it followed, and identify exactly where and why it went wrong. But the real value is what happens when the reasoning and retrieval graphs feed back into the system. And they feed back in two different ways, serving two different purposes.&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;pipeline planner&lt;/strong&gt; traverses the reasoning and retrieval graphs to decide &lt;strong&gt;what context the agent sees&lt;/strong&gt;. This is structural optimization. The planner can start from &lt;code&gt;kb:2201&lt;/code&gt; and traverse all evaluated edges pointing at it across every run. If it discovers that &lt;code&gt;kb:2201&lt;/code&gt; gets rejected 94% of the time for firmware queries, it deprioritizes or excludes it from the candidate set for that query type. It can walk &lt;code&gt;agent:support_tier2-&amp;gt;evaluated-&amp;gt;*&lt;/code&gt; to see every piece of evidence this agent has ever considered, grouped by action, filtered by verdict. It learns which retrieval patterns, graph paths, and scope filters correlate with correct outcomes and tightens the funnel accordingly.&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;LLM itself&lt;/strong&gt; receives relevant past reasoning edges as context to guide &lt;strong&gt;how it thinks about the candidates it receives&lt;/strong&gt;. This is behavioral optimization. When the agent retrieves context for a firmware issue on &lt;code&gt;product:x100&lt;/code&gt;, the system can also pull in past evaluated edges for queries in the same product category: "Last time an agent evaluated a firmware issue on this product line, it used &lt;code&gt;kb:4821&lt;/code&gt; (identical symptoms, same SKU) and rejected &lt;code&gt;kb:2201&lt;/code&gt; (wrong product line). That decision was correct." The agent is not just getting better candidates. It is getting guidance on how to reason about those candidates, grounded in what actually worked before.&lt;/p&gt;

&lt;p&gt;Those two mechanisms work on different layers. The pipeline planner tightens the funnel so fewer, better candidates reach the agent. The past reasoning edges sharpen how the agent evaluates whatever reaches it. Together, they compress errors from both directions. This is different from a context graph, which stores world facts (this customer owns this product). The reasoning and retrieval graphs extend the context graph with behavioral facts: how agents reasoned, what worked, what did not. The pipeline planner and the LLM both walk all three layers.&lt;/p&gt;

&lt;h3&gt;
  
  
  The graph feedback loop
&lt;/h3&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;This is the critical shift.&lt;/strong&gt; The reasoning and retrieval graphs turn retrieval from a static pipeline into a feedback loop. The funnel does not just narrow the data. It narrows itself. Every agent run produces decided, evaluated, and retrieval edges. Those edges inform the next pipeline plan. The scoping gets tighter. The graph paths get more precise. And because both graphs are written in the same transaction as the action, you have a causally consistent, traversable record of every decision the agent ever made, every piece of evidence it considered, and the retrieval path that produced it.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4bekos5qtp4z4jij8ah5.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4bekos5qtp4z4jij8ah5.png" alt="The graph feedback loop" width="800" height="727"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Let me walk through what this looks like concretely. On the first run for a given query type, the retrieval funnel starts with 2.8 million candidates and narrows to 14. The agent evaluates the top candidates, creating an evaluated edge for each one with a verdict and reason. It makes a decision. The decision turns out to be correct. The decided edge, the evaluated edges, and the retrieval edges all persist in the graph.&lt;/p&gt;

&lt;p&gt;On the next similar query, both mechanisms kick in. The pipeline planner traverses past retrieval and reasoning edges. It knows the successful graph path. It knows which filter combination produced a good reduction ratio. It knows &lt;code&gt;kb:2201&lt;/code&gt; was rejected last time for being the wrong product line, so it excludes it from the candidate set entirely. The funnel goes from 2.8 million to 9 instead of 14. Meanwhile, the agent receives past evaluated edges as context alongside the candidates: "For similar firmware queries, &lt;code&gt;kb:4821&lt;/code&gt; was used because it described identical symptoms on the same SKU. &lt;code&gt;kb:5540&lt;/code&gt; was rejected because the issue was already resolved." The agent is not just seeing better candidates. It is seeing how a previous agent successfully reasoned about similar candidates.&lt;/p&gt;

&lt;p&gt;After a hundred similar queries, the graph contains a rich web of evaluated edges for that query type. The planner can see, across all runs, which evidence records consistently get used versus rejected, which rejection reasons recur, and which evaluation patterns correlate with correct outcomes. The LLM, on every new run, gets the distilled reasoning patterns from the most relevant past decisions. The funnel is no longer exploring. The reasoning is no longer starting from scratch. Both layers are applying learned patterns. And the patterns are not summaries or statistics computed offline. They are live graph topology that gets traversed on every run.&lt;/p&gt;

&lt;p&gt;This is what "deterministic" means in practice. Not that the LLM produces identical tokens every time. It means that given the same type of query, the system follows the same learned retrieval pattern, produces the same quality of context, and arrives at the same caliber of decision. The variability gets squeezed out of the data layer, which is where it was causing problems in the first place. The model can still be creative where creativity helps. But the retrieval is locked down.&lt;/p&gt;

&lt;h3&gt;
  
  
  The full retrieval funnel
&lt;/h3&gt;

&lt;p&gt;Putting it all together, here is the complete pipeline from raw candidates to agent context, with graphs feeding back into pipeline planning. &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Feuk9mceyalc1m8w2nu4q.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Feuk9mceyalc1m8w2nu4q.png" alt="The full retrieval funnel" width="800" height="1099"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;After the agent acts, the decided, evaluated, and retrieval edges are written back, feeding into the next cycle of pipeline planning. The funnel self-tightens with every run.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why the graphs break in a fragmented stack
&lt;/h3&gt;

&lt;p&gt;There is one more thing worth making concrete, because it is easy to nod along with "you need one database" without feeling why.&lt;/p&gt;

&lt;p&gt;Say your reasoning and retrieval edges write to Postgres, your vector embeddings live in Pinecone, and your graph relationships are in Neo4j. The pipeline planner needs to query: "for this query type, which graph paths and filter combinations correlated with correct decisions in the last 200 runs?" That query needs to join reasoning edges and retrieval edges in Postgres with graph topology in Neo4j and vector metadata in Pinecone. What transaction boundary covers all three systems? There is not one. So the planner is reading behavioral data that may or may not reflect what actually happened, referencing graph paths that may have changed since the edges were written, and correlating against outcomes that landed in a different system at a different time. The feedback loop is broken before it starts. You are building a learning system on top of eventually consistent quicksand.&lt;/p&gt;

&lt;p&gt;That is why this architecture needs a multi-model graph database. Not because fewer databases is aesthetically nicer, but because the reasoning and retrieval edges need to live in the same graph as the entities they reference. The feedback loop is a graph traversal: walk from an agent through its decided edges, through its evaluated edges to each piece of evidence, follow the retrieval edges, compare outcomes. That only works when the reasoning and retrieval edges, the documents, the graph relationships, the vector embeddings, and the agent state all live in the same engine with the same transactional guarantees. SurrealDB handles all of this natively. This is the exact problem we designed it to solve.&lt;/p&gt;

&lt;h2&gt;
  
  
  Supporting research
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Cognitive Architectures for Language Agents (CoALA)&lt;/strong&gt; — Sumers, Yao, Narasimhan, Griffiths · 2024. Formalizes agent memory into episodic, semantic, and procedural types coordinated through a single decision loop. Reasoning and retrieval graphs map directly to CoALA's episodic memory: structured records of past actions that inform future decisions. The framework makes clear that scattering memory types across separate systems works against reliable agent behavior.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;From Local to Global: A Graph RAG Approach&lt;/strong&gt; — Edge, Trinh, Larson, Truitt (Microsoft Research) · 2024. Demonstrated that combining knowledge graphs with retrieval-augmented generation significantly outperforms vector-only retrieval for complex queries requiring multi-hop reasoning and structured context.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Retrieval-Augmented Generation: A Survey&lt;/strong&gt; — Gao, Xiong, Gao et al. · 2023. Identifies consistent failure modes in RAG at scale: top-k instability, chunk blending, precision degradation. Recommends constraining the candidate set before similarity search using structural filters and graph traversal.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Graph Retrieval-Augmented Generation: A Survey&lt;/strong&gt; — Peng, Zhu, Liu et al. · 2024. First comprehensive overview of GraphRAG, formalizing how graph-based indexing, retrieval, and generation improve accuracy and context-awareness over similarity-only approaches.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Near-perfect agent accuracy comes from two things working together. A retrieval pipeline that scopes candidates using structural filters, temporal constraints, graph traversal, and business rules before similarity ranking, all within a single ACID transaction. That gets you to roughly 90–95% on large corpora by eliminating the consistency gaps and retrieval noise that cause most agent failures.&lt;/p&gt;

&lt;p&gt;And reasoning and retrieval graphs stored as edges in the same transaction as the agent's action, feeding back into the pipeline so the funnel tightens with every run. That is what closes the gap to 99%+.&lt;/p&gt;

&lt;p&gt;Neither works if your data models are split across separate systems. The scope query cannot compose and the feedback loop breaks unless everything shares one transaction boundary. That is why we built SurrealDB.&lt;/p&gt;

&lt;p&gt;On the first run, 2.8 million candidates narrowed to 14. A few hundred runs later, it narrows to 9, then 6, each time more precise, each time learning from the last. That is what deterministic accuracy looks like in practice: agents that do not just perform well on average, but get better with every decision they make.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Sumers, T.R., Yao, S., Narasimhan, K., &amp;amp; Griffiths, T.L. (2024). Cognitive Architectures for Language Agents. &lt;em&gt;Transactions on Machine Learning Research&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;Edge, D., Trinh, H., Larson, J., &amp;amp; Truitt, S. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. &lt;em&gt;arXiv:2404.16130&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;Gao, Y., Xiong, Y., Gao, X., et al. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. &lt;em&gt;arXiv:2312.10997&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;Peng, B., Zhu, Y., Liu, Y., et al. (2024). Graph Retrieval-Augmented Generation: A Survey. &lt;em&gt;arXiv:2408.08921&lt;/em&gt;.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>surrealdb</category>
      <category>agents</category>
      <category>database</category>
    </item>
  </channel>
</rss>
