<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Sriprajna</title>
    <description>The latest articles on DEV Community by Sriprajna (@sriprajnaa).</description>
    <link>https://dev.to/sriprajnaa</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4148270%2Fb69eda7c-d1ff-48ff-a383-52fb993c99d1.png</url>
      <title>DEV Community: Sriprajna</title>
      <link>https://dev.to/sriprajnaa</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/sriprajnaa"/>
    <language>en</language>
    <item>
      <title>My Maintenance Agent Invented a Repair That Never Happened</title>
      <dc:creator>Sriprajna</dc:creator>
      <pubDate>Tue, 29 Sep 2026 01:22:56 +0000</pubDate>
      <link>https://dev.to/sriprajnaa/my-maintenance-agent-invented-a-repair-that-never-happened-42pb</link>
      <guid>https://dev.to/sriprajnaa/my-maintenance-agent-invented-a-repair-that-never-happened-42pb</guid>
      <description>&lt;p&gt;The agent told me UAV-204 had a shaft alignment correction after a bearing replacement. I checked the source reports. No such repair exists. The agent hadn't hallucinated from nowhere — it had picked the detail up from its own memory, which was carrying data from an earlier test run that I'd never cleared out.&lt;/p&gt;

&lt;p&gt;That single bad citation is what shaped the memory layer of the UAV fleet maintenance agent I built. This is the engineering story: what the system does, how I caught the fabrication, and the fix that made every citation checkable again.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What the system does&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Maintenance teams accumulate inspection reports, fault notes, component swaps, and technician observations. The useful part is usually buried: the last time this exact airframe, or this exact motor batch, had this exact problem, and what fixed it.&lt;/p&gt;

&lt;p&gt;The agent keeps that history per aircraft and per component, using &lt;a href="https://github.com/vectorize-io/hindsight" rel="noopener noreferrer"&gt;Hindsight agent memory&lt;/a&gt; as the storage and retrieval layer. When a new fault is reported, it does three things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Retain&lt;/strong&gt; each maintenance report in a memory bank, tagged with the date the event actually happened.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Recall&lt;/strong&gt; the history relevant to the reported aircraft and symptom.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Answer&lt;/strong&gt;, citing that evidence, and return the retrieved reports alongside the answer so a technician can check them.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The rule I held myself to: the agent should never state a repair, a date, or a finding that isn't actually in the retained reports. If it isn't there, the agent should say so.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Where it broke&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I'd been iterating on the same memory bank across several test runs — reloading data, tweaking the agent, asking the same test questions. What I didn't realize was that every one of those runs had been adding to the same bank, never clearing it.&lt;/p&gt;

&lt;p&gt;When I finally ran a clean set of test queries, the fabricated shaft-alignment repair for UAV-204 showed up in the output, dated correctly-looking but tied to an event that had never been logged. A second answer cited a repair on a date that didn't match anything in my dataset. The model wasn't lying so much as pattern-completing on top of a bank that had drifted away from ground truth — old test artifact's, partial seed runs, and leftover experiment data were all sitting in the same memory, and the model treated all of it as equally valid history.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Storing reports: use the event date, and use a clean bank&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The retain step itself was fine. Each report goes in with the date it actually happened, not the date I happened to load it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;python&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;store&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;date&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;retain&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;bank_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;BANK&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;timestamp&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fromisoformat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;date&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;isoformat&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
            &lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;Exception&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;   retry &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;What I'd missed was bank hygiene. Every reload and every test cycle had been writing into &lt;code&gt;uav-fleet-maintenance&lt;/code&gt;, and nothing ever removed the earlier, messier data. The fix wasn't a smarter prompt — it was starting over with a fresh bank ID (&lt;code&gt;uav-fleet-v2&lt;/code&gt;), reseeding from a single clean dataset, and treating that seeding step as a one-time, auditable event with a flag file so it can't silently run twice.&lt;/p&gt;

&lt;p&gt;*&lt;em&gt;Answering: recall first, then decide what the model is allowed to say&lt;br&gt;
*&lt;/em&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;python&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;ask&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;reflect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;bank_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;BANK&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;reflect&lt;/code&gt; does the retrieval and generation together, but I still pull the raw evidence separately, specifically so I can look at it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;python&lt;/span&gt;
&lt;span class="n"&gt;evidence&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;recall&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;bank_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;BANK&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;question&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That second call is what let me catch the fabrication in the first place. The final answer read smoothly and sounded exactly as confident whether it was right or wrong. The raw recall results didn't — they were the actual retained text, checkable sentence by sentence against my source data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How I check that it worked&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;After the fresh bank was seeded, I ran the same three test queries and checked every claim against the source dataset by hand:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A query about a new aircraft, 48 flight hours on a motor batch known for bearing wear in the 44–52 hour range: the answer now opened by naming that exact hour pattern and led with a bearing inspection, citing four real aircraft with real dates that matched my dataset.&lt;/li&gt;
&lt;li&gt;A query about intermittent vibration at low-to-mid throttle: the answer correctly prioritized checking motor mount screws first, citing the one case where a technician had replaced the motor before finding the real cause was loose screws — a genuinely useful "don't repeat this mistake" citation.&lt;/li&gt;
&lt;li&gt;A follow-up on a component with a known repeat-failure history: the answer recalled both prior incidents on that same unit, with matching dates, and noted that a partial fix hadn't held the first time.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;No invented repairs. No mismatched dates. The evidence panel I later added to the project's Streamlit interface makes this checking step something anyone can do in seconds, not just me digging through logs — the agent's answer sits next to the exact reports it pulled to write that answer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Lessons I'd reuse&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Never reuse a memory bank across test iterations without clearing it.&lt;/strong&gt; A bank isn't a scratchpad — treat every reload as a fresh start with a new bank ID, or you'll eventually get answers that blend real and stale data.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Always retrieve and inspect the raw evidence, separately from the final answer.&lt;/strong&gt; A fluent wrong answer and a fluent right answer look identical until you check what they actually pulled from memory.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pass the real event timestamp on every retained record.&lt;/strong&gt; Ordering is half the value of a maintenance history.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Make a seeding step idempotent and flagged.&lt;/strong&gt; A silent double-load is exactly the kind of thing that produces the fabrication I hit.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Surface the evidence to the end user, not just to yourself.&lt;/strong&gt; The people acting on this agent's advice are the ones who need to verify it, not just trust it.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Limits&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;This runs on a synthetic dataset — reports I wrote by hand to plant real patterns (bearing wear tied to a motor batch and flight-hour range, a mount-screw failure mode, a thermal issue) across a fleet of aircraft, not real technician logs. I haven't measured recall accuracy at real-fleet scale, and I'm not claiming this generalizes past what I tested. What I can say is that the specific failure mode that worried me — an agent stating something that never happened — is now visible, checkable, and the checking step is built into the product, not something I do by hand after the fact.&lt;/p&gt;

&lt;p&gt;Docs for the memory layer are here: &lt;a href="https://hindsight.vectorize.io/" rel="noopener noreferrer"&gt;Hindsight documentation&lt;/a&gt;, and a broader explanation of what agent memory is for is here: &lt;a href="https://vectorize.io/what-is-agent-memory" rel="noopener noreferrer"&gt;what is agent memory&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Code: [&lt;a href="https://github.com/njain2-del/Hindsight-UAV-Agent" rel="noopener noreferrer"&gt;https://github.com/njain2-del/Hindsight-UAV-Agent&lt;/a&gt;]&lt;/p&gt;

</description>
    </item>
  </channel>
</rss>
