<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Bhawani Shekhawat</title>
    <description>The latest articles on DEV Community by Bhawani Shekhawat (@theanarchytect).</description>
    <link>https://dev.to/theanarchytect</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4240894%2Fe89bc0c4-34d7-4884-93fa-23d94800edb9.png</url>
      <title>DEV Community: Bhawani Shekhawat</title>
      <link>https://dev.to/theanarchytect</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/theanarchytect"/>
    <language>en</language>
    <item>
      <title>RAG: The Lawyer Who Refuses to Guess</title>
      <dc:creator>Bhawani Shekhawat</dc:creator>
      <pubDate>Thu, 12 Nov 2026 13:58:07 +0000</pubDate>
      <link>https://dev.to/theanarchytect/rag-the-lawyer-who-refuses-to-guess-1ia9</link>
      <guid>https://dev.to/theanarchytect/rag-the-lawyer-who-refuses-to-guess-1ia9</guid>
      <description>&lt;p&gt;We smart-ified our phones, watches, fans, and homes. I’m genuinely concerned the coriander inside my refrigerator is about to demand Wi-Fi.&lt;/p&gt;

&lt;p&gt;Enter LLMs: the pinnacle of modern innovation, allowing us to automate the exact experience of talking to a guy who read half a Wikipedia article.&lt;/p&gt;

&lt;p&gt;Let’s put him on the stand.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyzn5o4y25ev4tk2k8izm.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyzn5o4y25ev4tk2k8izm.png" alt="Post Visual" width="800" height="437"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Act 1: The Know-It-All
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Courtroom.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Presiding: &lt;strong&gt;Judge Context Window&lt;/strong&gt;, a man with a strict limit on how many pages he’ll read before his eyes glaze over. (Remember him. He’s coming back.)&lt;/p&gt;

&lt;p&gt;Asking the questions: &lt;strong&gt;Counsel Footnote&lt;/strong&gt;, a lawyer who has never said a sentence without a source, and has footnotes on his footnotes.&lt;/p&gt;

&lt;p&gt;On the stand: the witness, &lt;strong&gt;an LLM&lt;/strong&gt;. You’ve worked with him. Every company has one: the colleague with an answer to everything, in every meeting, within four seconds. Tie straight. Self-doubt not found.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;His entire personality:&lt;/em&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;KnowItAll&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;IWitness&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="nc"&gt;Answer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;   
        &lt;span class="p"&gt;{&lt;/span&gt; 
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;LookUpFact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; 
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;FactNotFoundException&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="c1"&gt;// "I don't know" was never implemented&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;Improvise&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;confidence&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; 
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;State your expertise.&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;Everything&lt;/em&gt;.”&lt;br&gt;
&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Explain our payment service’s retry logic.&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;Exponential backoff with jitter, five attempts, defined in &lt;code&gt;RetryPolicy.cs&lt;/code&gt;.&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;We don’t have a &lt;code&gt;RetryPolicy.cs&lt;/code&gt;.&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;We do now.&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Have you ever worked at this company?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;No&lt;/em&gt;.”&lt;br&gt;
&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Read our architecture docs?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “No.”&lt;br&gt;
&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Attended the postmortem?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;No&lt;/em&gt;.”&lt;br&gt;
&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Then how do you know?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;I’m confident.&lt;/em&gt;”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;em&gt;In humans, this is called a &lt;strong&gt;promotion&lt;/strong&gt;. In LLMs, it’s called a &lt;strong&gt;hallucination&lt;/strong&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;He’s not lying, by the way. Lying requires knowing the truth. The witness is autocomplete with a confidence problem: he predicts the most plausible next word, and plausible has never once been a synonym for true.&lt;/p&gt;

&lt;p&gt;His training has a cutoff date. Your internal incident from three months ago may never have been part of his training data. That recent outage isn’t on the syllabus.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;Hallucination is just improv with a GPU.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  Act 2: The Intervention
&lt;/h2&gt;

&lt;p&gt;Counsel Footnote rises.&lt;/p&gt;

&lt;p&gt;“&lt;em&gt;Your Honor, we are not asking the witness to remember anything. We’re asking him to read the file.&lt;/em&gt;”&lt;/p&gt;

&lt;p&gt;“&lt;em&gt;Granted&lt;/em&gt;,” says &lt;strong&gt;Judge Context Window&lt;/strong&gt;. “Keep it short. I have a page limit.”&lt;/p&gt;

&lt;p&gt;That is &lt;strong&gt;RAG: Retrieval-Augmented Generation&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Three words. Three jobs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Retrieval&lt;/strong&gt;: Go find the relevant pages.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Augmented&lt;/strong&gt;: Staple them to the question.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Generation&lt;/strong&gt;: Now answer, from the file, not from feelings.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Notice what didn’t happen.&lt;/p&gt;

&lt;p&gt;The witness didn’t get smarter, wiser, or permanently enlightened. He read the file, answered, and by the next question he’s a goldfish with a law degree.&lt;/p&gt;

&lt;p&gt;In code, the entire intervention looks deceptively simple:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Before: the witness, alone with his confidence&lt;/span&gt;
&lt;span class="kd"&gt;var&lt;/span&gt; &lt;span class="n"&gt;testimony&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;witness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Answer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;// After: the witness, with homework&lt;/span&gt;
&lt;span class="kd"&gt;var&lt;/span&gt; &lt;span class="n"&gt;caseFile&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;clerk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;var&lt;/span&gt; &lt;span class="n"&gt;testimony&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;witness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Answer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;caseFile&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Same witness. One extra parameter. A much better basis for testimony.&lt;/p&gt;

&lt;p&gt;Nobody rewrote his training. We put evidence on his desk for this answer. Take it away, and unless the surrounding application keeps it available, he’s back to being a goldfish with a law degree.&lt;/p&gt;

&lt;p&gt;This is also why &lt;strong&gt;RAG and fine-tuning are not the same thing.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Fine-tuning changes the witness through further training. RAG changes what’s on his desk when you ask.&lt;/p&gt;

&lt;p&gt;One can teach him how your company likes its answers. The other puts your company’s actual documents in front of him.&lt;/p&gt;

&lt;p&gt;If the question is what your current deployment policy says, the current deployment policy would be a useful exhibit.&lt;/p&gt;

&lt;p&gt;Which brings us to that clerk.&lt;/p&gt;

&lt;p&gt;Everything interesting happens down the stairs.&lt;/p&gt;




&lt;h2&gt;
  
  
  Act 3: The Clerk in the Basement
&lt;/h2&gt;

&lt;p&gt;Picture the courthouse basement.&lt;/p&gt;

&lt;p&gt;Fluorescent lights. A coffee machine older than the case law. And one wall, floor to ceiling, covered in a giant corkboard: the detective-show kind, index cards and red string, minus the conspiracy.&lt;/p&gt;

&lt;p&gt;We’ll call it the &lt;strong&gt;Wall&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The clerk is not the witness, and he’s not AI magic. He’s ordinary software standing between the question and the witness. His job is to run downstairs and come back with the right pages.&lt;/p&gt;

&lt;p&gt;He has two specialists helping him:&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;Interpreter&lt;/strong&gt;, who translates words into numerical representations.&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;Senior Clerk&lt;/strong&gt;, who has developed a healthy intolerance for irrelevant paperwork.&lt;/p&gt;

&lt;p&gt;Together, they prepare one thing: the &lt;strong&gt;Case File&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Night Shift: Preparing the Evidence&lt;br&gt;
First delivery: the company’s 900-page wiki.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;“Your Honor, this isn’t evidence. It’s a hostage situation.”&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Judge Context Window refuses to read it.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;So the clerk starts cutting documents into smaller pieces. A few paragraphs per card. That’s &lt;strong&gt;chunking&lt;/strong&gt;.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The art is where you cut.&lt;/p&gt;

&lt;p&gt;A card gets read alone, like a WhatsApp screenshot: no context, maximum misunderstanding.&lt;/p&gt;

&lt;p&gt;Cut a joke between the setup and the punchline, retrieve only the second half, and the witness solemnly announces:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;“…and that’s why the chicken crossed the road.”&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;So the clerk lets neighbouring cards overlap. Each card repeats a little of the previous one.&lt;/p&gt;

&lt;p&gt;Think of it as the “Previously on…” recap. Nobody starts episode four cold.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;paragraphs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Count&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Card&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;paragraphs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Skip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Take&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Step forward four paragraphs. Read five. One paragraph of overlap.&lt;/p&gt;

&lt;p&gt;And the clerk doesn’t just keep the text. He retains its source, location, version, and access permissions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Counsel Footnote&lt;/strong&gt; does not accept “&lt;em&gt;found it somewhere&lt;/em&gt;” as a chain of custody.&lt;/p&gt;

&lt;p&gt;Now comes the &lt;strong&gt;Interpreter&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Not a language interpreter. This one converts text into a numerical representation called an &lt;strong&gt;embedding&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Think of it as giving each card an address based on aspects of its meaning.&lt;/p&gt;

&lt;p&gt;A keyword search might miss the connection because the words don’t match. The Interpreter sees that “&lt;em&gt;server down&lt;/em&gt;” and “&lt;em&gt;prod is on fire&lt;/em&gt;” are describing the same bad Tuesday.&lt;/p&gt;

&lt;p&gt;They end up near each other on the Wall. Strangers who share a smoking break.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Wall&lt;/strong&gt; is not a map of truth. It’s a map for finding related material.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;And the Wall, which stores these numerical representations and answers one question brilliantly - “what’s pinned near here?” - is a &lt;strong&gt;vector database&lt;/strong&gt;.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The clerk pins each card with its representation and metadata:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;foreach&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;var&lt;/span&gt; &lt;span class="n"&gt;card&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;library&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;SelectMany&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Chop&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;_wall&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Pin&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;card&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;_interpreter&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Embed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;card&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Content&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;card&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Metadata&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That’s the preparation work: chunking, embeddings, and indexing. Do it once, then repeat it whenever the underlying documents change.&lt;/p&gt;

&lt;p&gt;The basement is ready.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Day Shift: Someone Wants an Answer&lt;/strong&gt;&lt;br&gt;
Counsel Footnote has a question.&lt;/p&gt;

&lt;p&gt;The clerk doesn’t just grab the nearest document and sprint upstairs. There is a process.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Interpreter translates the question.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The question is converted into an embedding using the same model that prepared the documents. Otherwise, we’re comparing addresses from two different postal systems.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Wall retrieves candidates.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The vector database finds documents whose representations are close to the question’s representation. Twenty candidates come back. The Wall is enthusiastic. It thinks everything is relevant.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;The Senior Clerk intervenes.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;“Does this mention deployment?”&lt;br&gt;
“Yes.”&lt;/p&gt;

&lt;p&gt;“Does it answer who approves one?”&lt;br&gt;
“No.”&lt;/p&gt;

&lt;p&gt;“Then stop putting it on top.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The Senior Clerk is the &lt;strong&gt;reranker&lt;/strong&gt;. He takes the initial candidates, evaluates them against the actual question, and improves their order. Think of a dating app. It finds twenty people who are supposedly “&lt;strong&gt;98% compatible&lt;/strong&gt;.” The first dates establish whether any of them can actually hold a conversation. Fast and broad, then slower and sharper. The Senior Clerk keeps the strongest few. Twenty candidates become four.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Case File goes upstairs.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Judge Context Window looks at the four selected pieces of evidence.&lt;/p&gt;

&lt;p&gt;“Four. I can work with four.”&lt;/p&gt;

&lt;p&gt;Of course, four enormous documents can still get you thrown out. The Judge has a token budget, not a stationery allowance.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The witness answers.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The question and the Case File are passed to the LLM. The witness finally has something to read besides his own confidence.&lt;/p&gt;

&lt;p&gt;The whole retrieval operation looks something like this:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjjzu608whpluox94n5ye.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjjzu608whpluox94n5ye.png" alt="RAG Pipeline" width="572" height="1024"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The basement is the retrieval machinery. The courtroom is where the retrieved evidence meets generation.&lt;/p&gt;

&lt;p&gt;The witness speaks. The Wall does not.&lt;/p&gt;



&lt;p&gt;&lt;strong&gt;Act 4: Why Should I Care?&lt;/strong&gt;&lt;br&gt;
Because nobody ever woke up excited about a vector database.&lt;/p&gt;

&lt;p&gt;People are excited about a witness who finally works at their company.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Enterprise assistant&lt;/strong&gt;: Answers from your policies, not the internet’s. No more “per industry best practices” when your best practice is “ask Greg.”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Support bot&lt;/strong&gt;: Answers from actual product documentation, not product folklore.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Engineering assistant&lt;/strong&gt;: Has read your architecture docs and can point to the page. Unlike the architect.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Incident investigation&lt;/strong&gt;: Reads three years of postmortems nobody had time to re-read, and doesn’t sigh about it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Legal and patent research&lt;/strong&gt;: Searches your selected repository, not its own imagination.&lt;/p&gt;

&lt;p&gt;And here’s the underrated part.&lt;/p&gt;

&lt;p&gt;When a policy changes, you update the evidence in the retrieval system. You don’t need to retrain the underlying LLM every time someone edits a Confluence page.&lt;/p&gt;

&lt;p&gt;The paperwork still exists, of course. Update the document, replace or refresh the relevant index entries, and make sure the retrieval system is using the current version.&lt;/p&gt;

&lt;p&gt;The witness stays employed. The basement gets a ticket.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="c1"&gt;// The expensive way to learn the policy changed&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;RetrainFromScratch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;budget&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Series C"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;// The sensible way&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;_index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;ReindexAsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;updatedPolicy&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Retraining an LLM every time Confluence updates is like reprinting the dictionary because someone coined “rizz.”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Update the evidence, not the witness.&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  Act 5: The Reality Check
&lt;/h2&gt;

&lt;p&gt;Now Counsel Footnote cross-examines the file.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;RAG is not a hallucination exterminator.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Consider this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="kd"&gt;var&lt;/span&gt; &lt;span class="n"&gt;caseFile&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;clerk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="s2"&gt;"Who approves production deployments?"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;// Top hit: runbook_FINAL_v2_final(1).docx&lt;/span&gt;
&lt;span class="c1"&gt;// Last modified: 2019&lt;/span&gt;
&lt;span class="c1"&gt;// Contents: "All deployments require Dave's approval."&lt;/span&gt;
&lt;span class="c1"&gt;// Dave left in 2021.&lt;/span&gt;

&lt;span class="n"&gt;witness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Answer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;caseFile&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;// "Dave. Per the runbook."&lt;/span&gt;
&lt;span class="c1"&gt;// Citation: perfect. Accuracy: Dave.&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Chunk badly, and the paragraph containing the exception gets separated from the rule.&lt;/p&gt;

&lt;p&gt;Retrieve badly, and the witness reads the wrong folder with exactly the same confidence.&lt;/p&gt;

&lt;p&gt;Use stale documents, and you resurrect an approval chain whose final authority now runs a pottery studio.&lt;/p&gt;

&lt;p&gt;Garbage in, augmented garbage out.&lt;/p&gt;

&lt;p&gt;RAG doesn’t eliminate hallucination. Without evidence checks, it can give hallucination footnotes.&lt;/p&gt;

&lt;p&gt;And that brings us to the questions Counsel Footnote refuses to stop asking.&lt;/p&gt;

&lt;h3&gt;
  
  
  Was the witness allowed to read this?
&lt;/h3&gt;

&lt;p&gt;Permissions must be enforced before restricted documents reach the model.&lt;/p&gt;

&lt;p&gt;Not by politely instructing the LLM to ignore confidential information.&lt;/p&gt;

&lt;p&gt;The witness is not the security guard. We’ve established that already.&lt;/p&gt;

&lt;h3&gt;
  
  
  Does any of this actually answer the question?
&lt;/h3&gt;

&lt;p&gt;Four relevant documents do not necessarily constitute sufficient evidence.&lt;/p&gt;

&lt;p&gt;A document can mention deployment without identifying the person who approves it.&lt;/p&gt;

&lt;p&gt;“Then we’ve established that the search works,” says Counsel Footnote. “Not that the answer exists.”&lt;/p&gt;

&lt;p&gt;The application needs to assess whether the retrieved evidence actually supports the proposed answer.&lt;/p&gt;

&lt;p&gt;Something along these lines:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;caseFile&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;IsEmpty&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"I couldn't find relevant evidence."&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;evidenceReview&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Supports&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;caseFile&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"I found related documents, but not enough evidence to answer."&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No, &lt;code&gt;Supports()&lt;/code&gt; is not a magic truth detector. The engineering involves checking whether evidence supports the claim, whether sources contradict each other, and whether the system should abstain.&lt;/p&gt;

&lt;p&gt;The method name is short. The engineering isn’t.&lt;/p&gt;

&lt;p&gt;And a citation doesn’t end the cross-examination. It gives Counsel Footnote somewhere to start.&lt;/p&gt;

&lt;p&gt;Is this the current source?&lt;/p&gt;

&lt;p&gt;Does it actually support the claim?&lt;/p&gt;

&lt;p&gt;Is it authoritative?&lt;/p&gt;

&lt;p&gt;Appending the nearest document’s title to an answer is not accountability.&lt;/p&gt;

&lt;p&gt;It’s giving the improv troupe a bibliography.&lt;/p&gt;




&lt;h2&gt;
  
  
  Act 6: No Further Questions
&lt;/h2&gt;

&lt;p&gt;Back upstairs.&lt;/p&gt;

&lt;p&gt;Same courtroom. Same witness. Slightly less swagger.&lt;/p&gt;

&lt;p&gt;Counsel Footnote approaches the stand.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Who approves production deployments?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;Dave. According to the runbook.&lt;/em&gt;”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;When was that runbook last updated?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;2019&lt;/em&gt;.”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Does Dave still work here?&lt;/em&gt;”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The witness pauses.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;I don’t have current evidence confirming that.&lt;/em&gt;”&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Then who approves deployments today?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Witness&lt;/strong&gt;: “&lt;em&gt;I cannot establish that from the documents provided. The current approval policy would need to be checked.&lt;/em&gt;”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Judge Context Window looks up.&lt;/p&gt;

&lt;p&gt;Nobody invented an employee. Nobody fabricated a policy. Nobody produced a mysterious &lt;code&gt;RetryPolicy.cs&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The witness has finally produced something useful.&lt;/p&gt;

&lt;p&gt;A boundary. In code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;evidence&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Supports&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;claim&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;?&lt;/span&gt; &lt;span class="nc"&gt;Cite&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;claim&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;evidence&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Let me check."&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;RAG doesn’t automatically make the witness truthful. It gives you a way to put evidence in front of him — and build a process that holds his answers accountable.&lt;/p&gt;

&lt;p&gt;Intelligence was never about knowing everything.&lt;/p&gt;

&lt;p&gt;It’s knowing where the file is kept, and having the spine to say “let me check.”&lt;/p&gt;

&lt;p&gt;Counsel Footnote sits down.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “Your Honor, the witness has declined to guess.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Judge Context Window considers the evidence.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Judge Context Window&lt;/strong&gt;: “&lt;em&gt;On what grounds?&lt;/em&gt;”&lt;br&gt;
&lt;strong&gt;Counsel Footnote&lt;/strong&gt;: “&lt;em&gt;Insufficient evidence.&lt;/em&gt;”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A pause.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Judge Context Window&lt;/strong&gt;: “&lt;em&gt;&lt;strong&gt;Finally&lt;/strong&gt;. A supported conclusion.&lt;/em&gt;”&lt;/p&gt;

&lt;p&gt;No further questions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;#AnArchytect&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;u&gt;References&lt;/u&gt;&lt;/strong&gt;&lt;br&gt;
Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023). Court order addressing fictitious authorities submitted in a filing generated with ChatGPT.&lt;/p&gt;

&lt;p&gt;Microsoft Learn - Retrieval Augmented Generation (RAG) in Azure Machine Learning. Background on the RAG pattern and its components.&lt;/p&gt;

&lt;p&gt;Microsoft Learn - Develop a RAG Solution on Azure: Information Retrieval. Retrieval, ranking, and related implementation strategies.&lt;/p&gt;

&lt;p&gt;Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps in Retrieval-Augmented Generation. Research preprint, 2026.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>rag</category>
      <category>programming</category>
      <category>architecture</category>
    </item>
  </channel>
</rss>
