<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Gonzalo Blasco</title>
    <description>The latest articles on DEV Community by Gonzalo Blasco (@gonzoblasco).</description>
    <link>https://dev.to/gonzoblasco</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F585591%2F1db68b03-1e8a-4234-b8ca-a2c007b3e0a8.jpeg</url>
      <title>DEV Community: Gonzalo Blasco</title>
      <link>https://dev.to/gonzoblasco</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/gonzoblasco"/>
    <language>en</language>
    <item>
      <title>A local AI agent whose citations cannot be invented</title>
      <dc:creator>Gonzalo Blasco</dc:creator>
      <pubDate>Wed, 18 Nov 2026 23:56:36 +0000</pubDate>
      <link>https://dev.to/gonzoblasco/a-local-ai-agent-whose-citations-cannot-be-invented-5df1</link>
      <guid>https://dev.to/gonzoblasco/a-local-ai-agent-whose-citations-cannot-be-invented-5df1</guid>
      <description>&lt;p&gt;&lt;em&gt;Most RAG demos look great until the model invents a page number. This one refuses instead.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;I built a local-first agent that answers questions about contracts and invoices. It runs entirely on your laptop — no API key, no cloud, no data leaving the machine.&lt;/p&gt;

&lt;p&gt;The interesting part is not the retrieval. It's that &lt;strong&gt;the citations are checked by code, not trusted from the model.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem isn't retrieval
&lt;/h2&gt;

&lt;p&gt;Everyone builds a RAG in an afternoon. Upload PDFs, chunk, embed, search by similarity, pass the fragments to the model. It works, and the demo looks great.&lt;/p&gt;

&lt;p&gt;Then the model writes:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"The contract expires on December 15, according to clause 4.2, page 3."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The problem: clause 4.2 says something else, and page 3 doesn't exist. The model didn't lie on purpose. &lt;strong&gt;It filled a gap with something that sounded probable.&lt;/strong&gt; And since nobody checks, the wrong answer ships.&lt;/p&gt;

&lt;p&gt;That's not a retrieval problem. It's a &lt;strong&gt;verification&lt;/strong&gt; problem. And you don't fix it with a better model — you fix it with code.&lt;/p&gt;

&lt;h2&gt;
  
  
  The one idea
&lt;/h2&gt;

&lt;p&gt;Every answer comes back with structured citations: a &lt;code&gt;chunk_id&lt;/code&gt; and a literal quote. The app doesn't trust the model; it verifies it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;verify_citations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;citations&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;chunks_by_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;valid&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;invalid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;cit&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;citations&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;chunk&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;chunks_by_id&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cit&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"chunk_id"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;_normalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cit&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"quote"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;_normalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;valid&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cit&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;invalid&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cit&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;valid&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;invalid&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A citation is valid only if its quote is a &lt;strong&gt;substring&lt;/strong&gt; of the chunk, after whitespace normalization. If it fails, the model gets one more try. If it fails again, the answer becomes a refusal.&lt;/p&gt;

&lt;p&gt;This is not another LLM checking the first one. It's string containment, and anyone can reproduce it by opening the database.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three times it almost went wrong
&lt;/h2&gt;

&lt;p&gt;This is the part people usually skip.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. The embeddings were fake and the tests were green.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The first version called &lt;code&gt;ollama embed&lt;/code&gt;, a CLI subcommand that &lt;strong&gt;does not exist&lt;/strong&gt;. Every call failed and silently fell back to a three-dimension mock vector. The tests passed because they only checked structure, not values.&lt;/p&gt;

&lt;p&gt;The fix wasn't "improve the mock". It was removing the fallback entirely: if embeddings fail, ingestion fails loudly. A system that degrades silently is worse than one that breaks loudly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. The vector search wasn't vector search.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;sqlite-vec&lt;/code&gt; was never loaded. &lt;code&gt;chunks_vec&lt;/code&gt; was created as a regular table and the "KNN" was brute force. It worked by accident with 13 chunks and would have collapsed with 1,300.&lt;/p&gt;

&lt;p&gt;I found it by inspecting &lt;code&gt;sqlite_master&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;chunks_vec&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk_id&lt;/span&gt; &lt;span class="kt"&gt;INTEGER&lt;/span&gt; &lt;span class="k"&gt;PRIMARY&lt;/span&gt; &lt;span class="k"&gt;KEY&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;embedding&lt;/span&gt; &lt;span class="kt"&gt;BLOB&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;   ← &lt;span class="n"&gt;wrong&lt;/span&gt;
&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="no"&gt;VIRTUAL&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;chunks_vec&lt;/span&gt; &lt;span class="k"&gt;USING&lt;/span&gt; &lt;span class="nf"&gt;vec0&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;                          ← &lt;span class="k"&gt;right&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;3. Three times, a subagent reported files it never wrote.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;That's not a model problem, it's a trust problem: I accepted self-reports. The lesson was procedural. Before accepting any delivery, run the repo:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git status --short
uv run pytest -q
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If the file isn't on disk, it doesn't exist. Full stop.&lt;/p&gt;

&lt;h2&gt;
  
  
  The model is not the product
&lt;/h2&gt;

&lt;p&gt;Everyone asks "which model?". The honest answer: &lt;strong&gt;it matters less than you think, and more than you measure.&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Accuracy (18 questions)&lt;/th&gt;
&lt;th&gt;P50 latency&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 4 12B&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;18/18&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;49.5 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 3 4B&lt;/td&gt;
&lt;td&gt;4/18&lt;/td&gt;
&lt;td&gt;6.8 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The 4B is 7× faster and 4.5× less accurate. Fine for a demo. Not fine for a contract.&lt;/p&gt;

&lt;p&gt;And a detail nobody expects: &lt;strong&gt;a 9B can be slower than a 12B.&lt;/strong&gt; Qwen3.5:9B kept a 262k context resident and didn't finish a single answer in 90 seconds. The 12B answered in 50.&lt;/p&gt;

&lt;p&gt;The conclusion isn't "use the biggest model". It's &lt;strong&gt;measure on your machine, with your corpus, with your questions.&lt;/strong&gt; Someone else's benchmark is a prior, not a truth.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part the model doesn't do
&lt;/h2&gt;

&lt;p&gt;One of the 18 questions asked for the sum of two contract totals. The 12B failed it consistently: it can't add.&lt;/p&gt;

&lt;p&gt;The fix wasn't a better model. It was admitting arithmetic &lt;strong&gt;is not the LLM's job&lt;/strong&gt; and writing eight lines of deterministic code. The agent retrieves and composes; the calculator adds.&lt;/p&gt;

&lt;p&gt;Same principle as the citations, applied one layer down: &lt;strong&gt;what can be verified by code, is verified by code.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What local-first actually means
&lt;/h2&gt;

&lt;p&gt;Three proofs, not three promises:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Zero sockets.&lt;/strong&gt; During transcription or query, &lt;code&gt;lsof&lt;/code&gt; shows no outbound connection. Measured, not claimed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It can be deleted.&lt;/strong&gt; A button that removes index, embeddings and drafts — verified on the filesystem.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It isn't rented.&lt;/strong&gt; In ten years the archive opens without a subscription.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And a consequence almost nobody mentions: if the data doesn't leave, &lt;strong&gt;nothing comes in either&lt;/strong&gt;. The model has to be local because privacy isn't a feature — it's the architecture.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this project doesn't do
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;It doesn't read scanned PDFs. It rejects them with a clear message.&lt;/li&gt;
&lt;li&gt;It isn't fast. 50 seconds per answer with a 12B model.&lt;/li&gt;
&lt;li&gt;It doesn't scale. One person, one machine, one corpus.&lt;/li&gt;
&lt;li&gt;It doesn't have an independent benchmark. The 18 questions were written by the same person who wrote the system.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Saying this doesn't weaken the project. It makes it credible. A repo that admits its limits is a repo someone can use without surprises.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I built it
&lt;/h2&gt;

&lt;p&gt;It was planned as a step-based program, implemented by separate role agents (developer, designer, auditor, critic), and every claim in the docs was verified against the running code rather than accepted from a report. That process produced its own lesson, documented above: three times a delegate reported a module it never wrote, and the fix was procedural, not technical.&lt;/p&gt;

&lt;h2&gt;
  
  
  Origin
&lt;/h2&gt;

&lt;p&gt;This started as a design exercise for the &lt;strong&gt;DEV.to Hacktoberfest 2026 Weekend Challenge: Build for a Friend&lt;/strong&gt;. The official window closed on October 5, 2026 — after that, so it is &lt;strong&gt;not a submission&lt;/strong&gt;. The case study (“a freelance photographer with contracts she can't upload”) is a &lt;em&gt;persona&lt;/em&gt;, not a real person. It exists to give the design a constraint, not to claim a testimonial.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Code:&lt;/strong&gt; &lt;a href="https://github.com/gonzoblasco/localdoc-agent" rel="noopener noreferrer"&gt;github.com/gonzoblasco/localdoc-agent&lt;/a&gt; (MIT)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sibling project&lt;/strong&gt;, about people instead of contracts: &lt;a href="https://github.com/gonzoblasco/memoria-oral" rel="noopener noreferrer"&gt;github.com/gonzoblasco/memoria-oral&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The honest parts:&lt;/strong&gt; the &lt;em&gt;What went wrong&lt;/em&gt; section of the README, &lt;code&gt;eval/results.json&lt;/code&gt;, and the privacy check in &lt;code&gt;memoria-oral&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you build something similar, the one thing I'd steal is not the code: it's the habit of making the verifier independent from the thing being verified.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>rag</category>
      <category>opensource</category>
      <category>python</category>
    </item>
  </channel>
</rss>
