<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Aman Kumar</title>
    <description>The latest articles on DEV Community by Aman Kumar (@aman_k_).</description>
    <link>https://dev.to/aman_k_</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4175649%2F6f116a46-bfc8-4066-b770-1afb6af58a32.jpg</url>
      <title>DEV Community: Aman Kumar</title>
      <link>https://dev.to/aman_k_</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/aman_k_"/>
    <language>en</language>
    <item>
      <title>Ghostkeeper: a 1B local model that curates memories without rewriting the evidence</title>
      <dc:creator>Aman Kumar</dc:creator>
      <pubDate>Fri, 30 Oct 2026 07:39:27 +0000</pubDate>
      <link>https://dev.to/aman_k_/ghostkeeper-a-1b-local-model-that-curates-memories-without-rewriting-the-evidence-imo</link>
      <guid>https://dev.to/aman_k_/ghostkeeper-a-1b-local-model-that-curates-memories-without-rewriting-the-evidence-imo</guid>
      <description>&lt;p&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week4-2026-10-26"&gt;Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;You find an old recipe card. You can photograph it, back it up, and share it. None of that tells you who wrote it.&lt;/p&gt;

&lt;p&gt;An AI could give you a convincing answer. That is exactly the wrong thing to preserve.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Ghostkeeper is a local-first museum for objects and the accounts people attach to them.&lt;/strong&gt; Add a picture, an attributed memory, and the details you actually know. A small Gemma model selects passages worth keeping and suggests questions to ask next. You check the sources, approve the exhibit, and keep an HTML copy that opens offline.&lt;/p&gt;

&lt;p&gt;The important constraint: &lt;strong&gt;the model does not write the exhibit's quotations. The server retrieves the original words.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5b50a28r762tbpybxz7u.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5b50a28r762tbpybxz7u.png" alt="A real Gemma-curated exhibit with attributed quotations from a clearly fictional recipe-card account" width="720" height="648"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The walkthrough uses this short, explicitly fictional account:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Aaji kept this recipe card in a blue biscuit tin. I remember a yellow cloth around the jar. Nobody remembers who wrote the card.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The yellow cloth is a supplied memory. The writer is unknown. A useful curator should preserve that distinction, not turn the card into a beautifully written but invented biography.&lt;/p&gt;

&lt;p&gt;That is my interpretation of &lt;strong&gt;Ghost in the Machine&lt;/strong&gt;: remembrance without impersonation. No cloned relative, no simulated conversation with someone who died. Just their attributed words beside an object, with the gaps still visible.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Is this just a prompt that says "don't hallucinate"?&lt;/strong&gt; No. A prompt asks the model to behave. Ghostkeeper also limits what the model can contribute to the evidence: it selects IDs, while ordinary code controls which words become quotations. That does not make the recollection true. It makes the quotation traceable.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/uPzg9WJr7XM" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;The video follows one object through the actual app: entering its account, running local Gemma, opening a quote's source, reviewing the result, generating a stock ElevenLabs narrator, hearing a short sample, and opening the exported exhibit with the browser offline.&lt;/p&gt;

&lt;p&gt;The source material and illustration are fictional. The model request, narration request, and offline export are real. Footage is edited and retimed for clarity, so it is not a latency measurement.&lt;/p&gt;

&lt;p&gt;The application itself is &lt;strong&gt;not publicly hosted&lt;/strong&gt;. This walkthrough demonstrates it without exposing a private archive; the repository contains the code and fictional screenshots, not personal source material.&lt;/p&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/amangithub01" rel="noopener noreferrer"&gt;
        amangithub01
      &lt;/a&gt; / &lt;a href="https://github.com/amangithub01/ghostkeeper" rel="noopener noreferrer"&gt;
        ghostkeeper
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      A local-first personal museum: source-grounded Gemma curation, owner review, offline exhibits, and optional ElevenLabs narration.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Ghostkeeper&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;Objects survive. Their stories disappear. Save both.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Ghostkeeper is a local-first personal museum for the things people keep: a recipe card, an old radio, a letter, a photograph. Attach an attributed account, let a small local Gemma model select passages worth preserving, review the evidence, and keep an exhibit that opens offline.&lt;/p&gt;
&lt;p&gt;The project interprets Hacktoberfest 2026's &lt;strong&gt;Ghost in the Machine&lt;/strong&gt; theme as remembrance, not impersonation. It does not simulate a deceased person or invent missing family history. &lt;strong&gt;The model selects evidence; ordinary code retrieves the original words.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This repository contains the application, reproducible setup, tests, and reviewed fictional documentation images. It does &lt;strong&gt;not&lt;/strong&gt; contain API keys, a personal collection, raw recordings, video-editing projects, dependencies, or generated build output.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Read on:&lt;/strong&gt; &lt;a href="https://github.com/amangithub01/ghostkeeper#demo-video" rel="noopener noreferrer"&gt;Demo&lt;/a&gt; · &lt;a href="https://github.com/amangithub01/ghostkeeper#how-it-works" rel="noopener noreferrer"&gt;How it works&lt;/a&gt; · &lt;a href="https://github.com/amangithub01/ghostkeeper#run-locally" rel="noopener noreferrer"&gt;Run locally&lt;/a&gt; · &lt;a href="https://github.com/amangithub01/ghostkeeper#optional-elevenlabs-narration" rel="noopener noreferrer"&gt;Narration&lt;/a&gt; · &lt;a href="https://github.com/amangithub01/ghostkeeper#architecture-and-code-map" rel="noopener noreferrer"&gt;Architecture&lt;/a&gt; · &lt;a href="https://github.com/amangithub01/ghostkeeper#privacy-and-storage" rel="noopener noreferrer"&gt;Privacy&lt;/a&gt; · &lt;a href="https://github.com/amangithub01/ghostkeeper#development-and-tests" rel="noopener noreferrer"&gt;Tests&lt;/a&gt; · &lt;a href="https://github.com/amangithub01/ghostkeeper#current-limitations" rel="noopener noreferrer"&gt;Limitations&lt;/a&gt;&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;Demo video&lt;/h2&gt;
&lt;/div&gt;
&lt;p&gt;&lt;a href="https://github.com/amangithub01/ghostkeeper/releases/download/v0.1.0/Ghostkeeper-Product-Demo.mp4" rel="noopener noreferrer"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Famangithub01%2Fghostkeeper%2FHEAD%2Fdocs%2Fimages%2Fghostkeeper-demo-thumbnail.png" alt="Watch the Ghostkeeper product demo: a fictional recipe card, real local Gemma curation, and ElevenLabs narration"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;a href="https://github.com/amangithub01/ghostkeeper/releases/download/v0.1.0/Ghostkeeper-Product-Demo.mp4" rel="noopener noreferrer"&gt;Watch or download the 2:14 demo (1080p MP4)&lt;/a&gt;&lt;/strong&gt;…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/amangithub01/ghostkeeper" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;The repository includes setup instructions, a credential-free environment template, focused unit tests, and Playwright acceptance tests. The code is public for inspection, but the project does not yet have a license.&lt;/p&gt;

&lt;p&gt;To run it locally:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/amangithub01/ghostkeeper.git
&lt;span class="nb"&gt;cd &lt;/span&gt;ghostkeeper
npm ci
ollama pull gemma3:1b
&lt;span class="nb"&gt;cp&lt;/span&gt; .env.example .env.local
npm run dev
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Requires Node.js 22.18 or newer and Ollama. Open &lt;code&gt;http://127.0.0.1:3000&lt;/code&gt;. Ollama must be running. ElevenLabs settings are optional; the local curation and export journey does not depend on them.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;h3&gt;
  
  
  A local core, with a separate cloud branch
&lt;/h3&gt;

&lt;p&gt;The interface is Next.js, React, TypeScript, and Tailwind. A local Next.js API validates inputs, calls Ollama, checks the result, and writes the collection to a local JSON file.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9avtmsifrbhdirmz2pm0.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9avtmsifrbhdirmz2pm0.png" alt="Implemented architecture: local browser, API, source validation, Ollama and Gemma, local archive and offline export; a separate consent-gated ElevenLabs branch" width="800" height="460"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The main path is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Object + attributed source
  -&amp;gt; numbered evidence passages
  -&amp;gt; local Gemma selects passage IDs and suggests questions
  -&amp;gt; server resolves IDs to original source text
  -&amp;gt; validation
  -&amp;gt; owner review and approval
  -&amp;gt; offline exhibit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;ElevenLabs sits on a separate, optional branch after approval and explicit sharing consent.&lt;/p&gt;

&lt;p&gt;The code follows those boundaries:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Responsibility&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;createArtifact()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Validate the object and attach its supplied source.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;evidencePassages()&lt;/code&gt; and &lt;code&gt;curate()&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Prepare evidence, call local Gemma, and resolve its selections.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;validateExhibit()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Check quotations against their cited source text.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;store.create()&lt;/code&gt; and &lt;code&gt;store.update()&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Persist local records and reject stale revisions.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;narrationText()&lt;/code&gt; and &lt;code&gt;generateNarration()&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Build an approved script and make the consent-gated speech request.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;exportHtml()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Create the portable, self-contained exhibit.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;localRequest()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Reject non-loopback hosts and cross-origin browser requests.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  The first version failed for a useful reason
&lt;/h3&gt;

&lt;p&gt;Initially, I asked the small model to return quotations and source identifiers. A real fictional-radio example failed with an &lt;strong&gt;ungrounded quote&lt;/strong&gt; error.&lt;/p&gt;

&lt;p&gt;The validator was doing its job. A quote that is nearly the source is not an exact quotation, and a copied identifier that is nearly correct is not the right source.&lt;/p&gt;

&lt;p&gt;I could have loosened the check or quietly substituted a hand-written exhibit. Instead, I changed the job I was giving the model.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Before:&lt;/strong&gt; select the evidence, copy its text, and reproduce the source identifier.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Now:&lt;/strong&gt; select a short passage ID. The server does the copying.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;evidencePassages()&lt;/code&gt; uses &lt;code&gt;Intl.Segmenter&lt;/code&gt; to split source text into sentence-sized passages. Longer passages are split into chunks of at most 1,000 UTF-16 code units, without dividing a surrogate pair. Each gets a simple ID such as &lt;code&gt;P1&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;For the recipe-card example:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;ID&lt;/th&gt;
&lt;th&gt;Original passage&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;P1&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Aaji kept this recipe card in a blue biscuit tin.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;P2&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;I remember a yellow cloth around the jar.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;P3&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Nobody remembers who wrote the card.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;curate()&lt;/code&gt; calls Ollama's &lt;code&gt;/api/chat&lt;/code&gt; with the configured &lt;code&gt;gemma3:1b&lt;/code&gt; model. Its JSON schema limits selections to the supplied passage IDs. The model is asked for one to four distinct selections and a small set of interview questions. It is not asked to transcribe quotes or reproduce the archive's source UUIDs.&lt;/p&gt;

&lt;p&gt;Then the server resolves the selections:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;quotes&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;selection&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;passageIds&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;unknown&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;passage&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;passages&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;find&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;passage&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;AppError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Unknown evidence selection.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;502&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;sourceId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;passage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sourceId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;quote&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;passage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;quote&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;validateExhibit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;quotes&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;questions&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;selection&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;questions&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="nx"&gt;artifact&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sources&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nx"&gt;model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;em&gt;Excerpt shortened for readability; the repository contains the full validation and error messages.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Unknown IDs, duplicate selections, and invalid output fail rather than turning into a convincing-looking fallback. &lt;code&gt;validateExhibit()&lt;/code&gt; separately checks that every resolved quote is an exact, continuous substring of its cited source.&lt;/p&gt;

&lt;p&gt;The model has less clerical work to get wrong. More importantly, it no longer has authority to rewrite the evidence.&lt;/p&gt;

&lt;h3&gt;
  
  
  The source is one click away
&lt;/h3&gt;

&lt;p&gt;Each quotation keeps its source attribution. &lt;strong&gt;View source&lt;/strong&gt; opens the original account, not another model-generated explanation.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq9u7956u5m68fnd30qig.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq9u7956u5m68fnd30qig.png" alt="The original fictional source, including the sentence that says the writer is unknown" width="720" height="648"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The panel titled &lt;strong&gt;"The story isn't finished"&lt;/strong&gt; contains optional questions. It is not a failed generation, and answering every question is not a prerequisite for export.&lt;/p&gt;

&lt;p&gt;There is a deliberate limit here: exact-quote checking establishes a relationship between two strings, not between a memory and historical reality. A mistaken account can be quoted perfectly.&lt;/p&gt;

&lt;p&gt;Questions remain model output, too. In this capture, Gemma asked what the blue tin might suggest about the card's history. That could encourage inference beyond the account. A prompt requesting non-leading questions does not guarantee neutral questions; the owner should skip or rewrite ones that do not fit. The app does not promote those questions into evidence.&lt;/p&gt;

&lt;h3&gt;
  
  
  Approval belongs to a version, not an object forever
&lt;/h3&gt;

&lt;p&gt;An owner must review an exhibit before exporting or narrating it. Adding another source clears the old exhibit and its approval.&lt;/p&gt;

&lt;p&gt;That matters more than a reassuring badge. If a second person adds a different recollection, the first person's review should not silently approve the new material.&lt;/p&gt;

&lt;p&gt;The local store serializes writes within one process, writes to a temporary file before renaming it, and checks revisions so a stale browser window cannot overwrite a newer update unnoticed. This is a single-process local archive, not a distributed database.&lt;/p&gt;

&lt;h3&gt;
  
  
  ElevenLabs: a narrator, not a relative
&lt;/h3&gt;

&lt;p&gt;After review, the owner can explicitly permit a narration request. &lt;code&gt;narrationText()&lt;/code&gt; builds a script from the title, source attribution, and selected quotations, with a reminder that these are attributed memories rather than independently verified facts.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;generateNarration()&lt;/code&gt; sends that text to ElevenLabs using a stock narrator and returns an MP3. It does not send the photograph or the full archive.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fk64r2ipe00uwpup7wuvu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fk64r2ipe00uwpup7wuvu.png" alt="A genuine generated narration player and MP3 download below the explicit sharing-consent controls" width="720" height="648"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The new walkthrough includes both a real generation and a short sample of the returned audio. Its explanatory voiceover also uses ElevenLabs' stock Sarah voice. No voice was cloned.&lt;/p&gt;

&lt;p&gt;The key stays server-side. Provider errors distinguish missing configuration, authentication, voice access, permissions, quota, and network failures without displaying credentials or raw provider responses. Paid synthesis requests are not automatically retried.&lt;/p&gt;

&lt;p&gt;Audio is temporary in the browser unless downloaded. It is &lt;strong&gt;not&lt;/strong&gt; embedded in the HTML exhibit.&lt;/p&gt;

&lt;h3&gt;
  
  
  The last step works without the app
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;exportHtml()&lt;/code&gt; produces a self-contained HTML file containing the image, selected quotations, questions, attribution, and &lt;strong&gt;full original source text&lt;/strong&gt;. No JavaScript, remote fonts, or running Ghostkeeper server is required to read it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxv32fb8uy9ylvqg11hg3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxv32fb8uy9ylvqg11hg3.png" alt="The exported exhibit opened with the browser offline, showing the original source" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This is portability, not a full archive backup/import format. It also creates a sharing responsibility: somebody receiving the export receives the full sources, not just the passages visible near the top.&lt;/p&gt;

&lt;h3&gt;
  
  
  What I actually checked
&lt;/h3&gt;

&lt;p&gt;The existing unit suite currently has &lt;strong&gt;16 passing tests&lt;/strong&gt;. The evidence-selection tests use controlled model responses to exercise the contract; they are not a benchmark of Gemma's judgement.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Check&lt;/th&gt;
&lt;th&gt;What the evidence supports&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Exact source resolution&lt;/td&gt;
&lt;td&gt;A selected passage resolves to the original text; extra invented model prose is not used as evidence.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Invalid selections&lt;/td&gt;
&lt;td&gt;Unknown, duplicate, and empty passage selections are rejected.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Multiple sources and Unicode&lt;/td&gt;
&lt;td&gt;Attribution survives passage splitting, including long text containing surrogate pairs.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;New evidence&lt;/td&gt;
&lt;td&gt;Adding a source clears the previous exhibit and approval.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Export and local storage&lt;/td&gt;
&lt;td&gt;Approval gates, escaping, concurrent saves, stale revisions, reload, and corruption preservation have focused checks.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Actual recipe walkthrough&lt;/td&gt;
&lt;td&gt;Local Gemma selected three quotes; all three matched the fictional source exactly. The narration played, and the HTML opened offline.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The &lt;strong&gt;four passing Playwright tests&lt;/strong&gt; cover the real local curation journey, reload and deletion, approval invalidation, mobile layout, request guards, and narration-setting diagnostics. They do not make paid ElevenLabs requests.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I have not established:&lt;/strong&gt; model-selection quality across a representative dataset, historical accuracy, family-user satisfaction, or superiority to another model. A successful demonstration and deterministic contract tests cannot justify those claims.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;For Ghostkeeper, the useful property is not simply that a model is small. It is that the curator can run where the source material lives.&lt;/p&gt;

&lt;p&gt;After downloading the model, curation does not need a hosted language-model API. That is a meaningful choice for a letter or recollection someone wants to preserve but does not want to upload.&lt;/p&gt;

&lt;p&gt;Gemma's open weights are available under Google's terms; this project uses the existing model, not a fine-tune. The model setting and prompts are visible, and the source-validation code can be inspected independently of the model. Local inference still uses the owner's hardware and electricity; "local" is not a claim of zero cost.&lt;/p&gt;

&lt;p&gt;The second form of independence is simpler: plain HTML. The approved exhibit can outlive a provider account or the running app.&lt;/p&gt;

&lt;p&gt;HTML export does not require an open model; it is a separate portability choice. Together, local inference and a readable export reduce two dependencies: where an account must be processed, and which service must remain available to read it.&lt;/p&gt;

&lt;h2&gt;
  
  
  My Agent Session
&lt;/h2&gt;

&lt;p&gt;I built and iterated on Ghostkeeper with &lt;strong&gt;GitHub Copilot CLI&lt;/strong&gt;. This section is a process summary, not a published session replay.&lt;/p&gt;

&lt;p&gt;The useful work was not a single prompt that produced a finished application. It was the loop between implementation, a real failure, and a narrower fix:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Build the local collection, source attribution, and offline export without adding a hosted database.&lt;/li&gt;
&lt;li&gt;Connect Gemma and reproduce the ungrounded-quotation failure on a fictional radio account.&lt;/li&gt;
&lt;li&gt;Keep the validator strict, change model output to passage IDs, and add regression cases for unknown selections, multiple sources, and Unicode.&lt;/li&gt;
&lt;li&gt;Diagnose narration configuration separately from provider authentication, voice access, and quota failures.&lt;/li&gt;
&lt;li&gt;Exercise the real app and record a fictional example instead of publishing somebody's family archive.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The &lt;a href="https://github.com/amangithub01/ghostkeeper" rel="noopener noreferrer"&gt;implementation and tests are public&lt;/a&gt;. No credentials or private collection data are included in the repository or demo.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;These are the partner categories supported by the implemented work:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Category&lt;/th&gt;
&lt;th&gt;Implemented use&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Real local &lt;code&gt;gemma3:1b&lt;/code&gt; inference through Ollama for passage selection and questions.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Best Use of ElevenLabs&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Consent-gated stock-voice exhibit narration, plus the demonstration's voiceover.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Best Use of GitHub Copilot&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Copilot CLI helped implement, debug, and iterate on the application and demonstration.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Ollama, Next.js, React, and TypeScript support the implementation; they are not extra partner entries. Render and DigitalOcean are &lt;strong&gt;not deployed or connected&lt;/strong&gt;. No additional sponsor integration is claimed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits I Would Rather State Than Hide
&lt;/h2&gt;

&lt;p&gt;This is a single-user local prototype. It accepts typed or pasted text; it does not record interviews, transcribe audio, recognize handwriting, research the web, or independently verify history.&lt;/p&gt;

&lt;p&gt;The archive is not encrypted by the app. Protect the machine and its backups. The server binds to loopback and checks request hosts and origins, but that is not public-service authentication. Do not expose it through a public tunnel or deploy it unchanged.&lt;/p&gt;

&lt;p&gt;Deleting an object does not remove exported files, downloaded MP3s, backups, or text already sent to a provider.&lt;/p&gt;

&lt;p&gt;The next work I would prioritize is evaluating question quality with people, handling contradictory accounts without merging them into a single "truth," and making backup and restore less manual. Those are next steps, not completed features.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Spirit of It
&lt;/h2&gt;

&lt;p&gt;In the fictional demo, the model selected this sentence:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Nobody remembers who wrote the card.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Ghostkeeper did not turn that absence into an answer. The sentence remained part of the exhibit, next to an invitation to ask another question.&lt;/p&gt;

&lt;p&gt;That is the small moment this prototype is built around: finding something worth asking while there may still be someone who can answer.&lt;/p&gt;

&lt;p&gt;A photo answers, "What did the object look like?"&lt;/p&gt;

&lt;p&gt;An attributed account can answer, "What did it mean to someone?"&lt;/p&gt;

&lt;p&gt;Ghostkeeper puts them together, without pretending that an unknown writer has suddenly become known.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>gemma</category>
      <category>ai</category>
    </item>
    <item>
      <title> Refund Sloth 🦥: Local AI Agents That Get Your Money Back While You Nap</title>
      <dc:creator>Aman Kumar</dc:creator>
      <pubDate>Sat, 24 Oct 2026 05:05:22 +0000</pubDate>
      <link>https://dev.to/aman_k_/refund-sloth-local-ai-agents-that-get-your-money-back-while-you-nap-3ee8</link>
      <guid>https://dev.to/aman_k_/refund-sloth-local-ai-agents-that-get-your-money-back-while-you-nap-3ee8</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week3-2026-10-19"&gt;Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Somewhere between 48% and 79% of us have paid for a free trial we forgot to cancel. The average person loses &lt;strong&gt;$187–$540 a year&lt;/strong&gt; to forgotten trials, missed return windows and subscriptions they stopped using. We &lt;em&gt;think&lt;/em&gt; we spend about $86 a month on subscriptions; it's closer to $219. And 44% of people keep a subscription simply because cancelling is a chore.&lt;/p&gt;

&lt;p&gt;That's the most sloth-shaped problem I know: the money is recoverable, but only if you do boring admin before a deadline.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Refund Sloth 🦥&lt;/strong&gt; does the boring part. You drop receipts, order emails and your bank statement into one folder (or let it read your mailbox) and go to sleep. At 2 AM a team of six local AI agents wakes up:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;🔭 &lt;strong&gt;Scout&lt;/strong&gt; finds new files through the official filesystem MCP server, locked to the inbox.&lt;/li&gt;
&lt;li&gt;📖 &lt;strong&gt;Reader&lt;/strong&gt; reads receipt &lt;em&gt;photos&lt;/em&gt; with a vision model, and PDFs and emails with a text model, into strict JSON.&lt;/li&gt;
&lt;li&gt;🦅 &lt;strong&gt;Deadline Hawk&lt;/strong&gt; works out return-by, trial-end, warranty and renewal dates and creates calendar reminders.&lt;/li&gt;
&lt;li&gt;🧾 &lt;strong&gt;Auditor&lt;/strong&gt; finds recurring charges in your bank CSV and flags 🧟 zombie, 👯 duplicate and 📈 price-hiked subscriptions.&lt;/li&gt;
&lt;li&gt;👻 &lt;strong&gt;Ghostwriter&lt;/strong&gt; drafts the cancellation and return emails.&lt;/li&gt;
&lt;li&gt;🦥 &lt;strong&gt;Sloth Boss&lt;/strong&gt; reviews everything with tool calls and writes your morning brief.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In the morning you get a notification:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;🦥 3 deadlines this week · $86.96/mo you can cut · 10 drafts ready&lt;br&gt;
Most urgent: ⏳ Peacock Premium free trial ends 2026-10-23 ($7.99 at stake).&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;You open the dashboard, click &lt;strong&gt;Approve &amp;amp; copy&lt;/strong&gt;, paste, send. Or click &lt;strong&gt;Cancel now ↗&lt;/strong&gt; to jump straight to the store's cancel page. Total effort: about 30 seconds. The sloth never sends or pays anything itself: a human stays in the loop, always.&lt;/p&gt;

&lt;p&gt;Even lazier extras:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;📬 &lt;strong&gt;Connect mail (optional):&lt;/strong&gt; give it a Gmail, iCloud or Outlook &lt;em&gt;app password&lt;/em&gt; and it fetches order, trial and renewal emails by itself. The mailbox is opened &lt;strong&gt;read-only&lt;/strong&gt;, and the password is stored encrypted on your machine (macOS Keychain / Windows DPAPI).&lt;/li&gt;
&lt;li&gt;🚨 &lt;strong&gt;Urgent popup:&lt;/strong&gt; anything ending in the next 2 days gets its own desktop notification and a red banner at the top of the dashboard.&lt;/li&gt;
&lt;li&gt;↗️ &lt;strong&gt;Cancel now:&lt;/strong&gt; every trial, subscription and return has a button that opens the store's cancel or return page. No more hunting for the hidden cancel link.&lt;/li&gt;
&lt;li&gt;🖥️ Runs on &lt;strong&gt;macOS and Windows&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Who gets to be lazier?&lt;/strong&gt; Anyone with a stack of receipts in a drawer, a "free" trial they meant to cancel, and a gym membership they've been "about to use again" since March.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/hmQlxosJAlk" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fd5lck471u4hzexcn8a9a.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fd5lck471u4hzexcn8a9a.gif" alt="Refund Sloth demo: landing page, dashboard tabs, agent log and mail connect" width="599" height="375"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Today:&lt;/strong&gt; the urgent banner, the morning brief, and money saved vs. money at risk.&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flkfysaf1ga5scqukx85x.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flkfysaf1ga5scqukx85x.png" alt="Today tab" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Subscriptions:&lt;/strong&gt; zombies, duplicates and price hikes, each with a Cancel link.&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftkiu8eqjdbdyeis8agcd.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftkiu8eqjdbdyeis8agcd.png" alt="Subscriptions tab" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Action queue:&lt;/strong&gt; ready-to-send emails. Nothing leaves without your click.&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0n741fkf833yzkuptbys.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0n741fkf833yzkuptbys.png" alt="Action queue" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;All data in the demo is fake (generated fixtures). No real receipts or bank statements were harmed.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/amangithub01" rel="noopener noreferrer"&gt;
        amangithub01
      &lt;/a&gt; / &lt;a href="https://github.com/amangithub01/refund-sloth" rel="noopener noreferrer"&gt;
        refund-sloth
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      🦥 Local AI agents that get your money back while you nap. Open-weight models (Ollama) + MCP, no API keys, no cloud.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;🦥 Refund Sloth&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;Local AI agents that get your money back while you nap.&lt;/strong&gt; Drop receipts, order emails and bank statements into a folder. Overnight, a team of open-weight models running on &lt;em&gt;your&lt;/em&gt; laptop finds forgotten free trials, closing return windows, expiring warranties and zombie subscriptions, then drafts the cancellation and return emails for you. In the morning you click &lt;strong&gt;Approve&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;No API keys. No cloud. $0 per run. Works with Wi-Fi off.&lt;/p&gt;
&lt;p&gt;🌐 &lt;strong&gt;Try the live demo:&lt;/strong&gt; &lt;a href="https://refund-sloth.vercel.app" rel="nofollow noopener noreferrer"&gt;refund-sloth.vercel.app&lt;/a&gt;. It shows the dashboard with sample data, runs entirely in your browser, and needs no install. The real agents and mail sync run only on your own machine (see &lt;a href="https://github.com/amangithub01/refund-sloth#setup" rel="noopener noreferrer"&gt;Setup&lt;/a&gt;).&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;🎬 Demo&lt;/h2&gt;
&lt;/div&gt;
&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/amangithub01/refund-sloth/docs/media/refund-sloth-demo.gif"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Famangithub01%2Frefund-sloth%2FHEAD%2Fdocs%2Fmedia%2Frefund-sloth-demo.gif" alt="Refund Sloth demo: landing page, laptop orbit, dashboard tabs, agent log, mail connect"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;▶️ Full-quality video: &lt;a href="https://github.com/amangithub01/refund-sloth/docs/media/refund-sloth-demo.mp4" rel="noopener noreferrer"&gt;docs/media/refund-sloth-demo.mp4&lt;/a&gt; (26 s, demo data, fake "today" = 2026-10-21)&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;📸 Screenshots&lt;/h2&gt;
&lt;/div&gt;
&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Landing&lt;/th&gt;
&lt;th&gt;Scroll: the laptop orbit&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/amangithub01/refund-sloth/docs/screenshots/landing.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Famangithub01%2Frefund-sloth%2FHEAD%2Fdocs%2Fscreenshots%2Flanding.png" alt="Landing page"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/amangithub01/refund-sloth/docs/screenshots/landing-orbit.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Famangithub01%2Frefund-sloth%2FHEAD%2Fdocs%2Fscreenshots%2Flanding-orbit.png" alt="Everything the sloth found, orbiting the laptop"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Today: urgent banner + morning brief&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Deadlines&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/amangithub01/refund-sloth/docs/screenshots/today.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Famangithub01%2Frefund-sloth%2FHEAD%2Fdocs%2Fscreenshots%2Ftoday.png" alt="Today"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/amangithub01/refund-sloth/docs/screenshots/deadlines.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Famangithub01%2Frefund-sloth%2FHEAD%2Fdocs%2Fscreenshots%2Fdeadlines.png" alt="Deadlines"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Subscriptions: zombies, duplicates, price hikes&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Action queue: drafts you approve&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;…&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/amangithub01/refund-sloth" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Every model runs locally through &lt;a href="https://ollama.com" rel="noopener noreferrer"&gt;Ollama&lt;/a&gt;.&lt;/strong&gt; No API keys anywhere in the project.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;Open-weight model&lt;/th&gt;
&lt;th&gt;Why&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Reader (images)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;qwen2.5vl:7b&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;small, fast vision model; reads crumpled receipt photos&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Reader (text), Hawk, Auditor, Ghostwriter, Boss&lt;/td&gt;
&lt;td&gt;&lt;code&gt;qwen3.5:35b&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;strong JSON output and tool calling; runs well on an M3 Max&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lite profile&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;qwen3:8b&lt;/code&gt; + &lt;code&gt;qwen2.5vl:3b&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;for laptops with 8–16 GB of RAM&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;MCP is the backbone.&lt;/strong&gt; I wrote five small MCP servers in TypeScript over stdio:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;ledger&lt;/code&gt; (SQLite)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;docs&lt;/code&gt; (PDF, image, EML, MBOX and CSV parsing)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;calendar&lt;/code&gt; (&lt;code&gt;.ics&lt;/code&gt; files with alarms)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;notify&lt;/code&gt; (desktop notifications and the brief)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;mail&lt;/code&gt; (read-only IMAP)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Alongside them runs the official &lt;code&gt;@modelcontextprotocol/server-filesystem&lt;/code&gt;, rooted at the inbox with only its read-only tools exposed. A small harness (&lt;code&gt;McpHub&lt;/code&gt;) starts every server, turns their tools into Ollama tool definitions and runs the agent loops. The full architecture diagram is in the README.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Hybrid by design: rules for math, models for mess.&lt;/strong&gt; The biggest lesson: don't let an LLM do arithmetic on your money.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Plain code:&lt;/strong&gt; hashing and dedup, all deadline math (document date, then &lt;code&gt;policies.json&lt;/code&gt;, then a clearly marked AI guess), recurring-charge detection, the zombie, duplicate and price-hike flags, and every number in the brief. All unit-tested.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;LLMs:&lt;/strong&gt; reading messy documents, guessing return policies for unknown stores, naming bank descriptors (&lt;code&gt;APPLE.COM/BILL 866-712-7753 CA&lt;/code&gt; → "Apple Music"), writing emails, and the Boss's review.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Structured output that doesn't break.&lt;/strong&gt; Every LLM call uses a zod schema, converted to JSON Schema and passed to Ollama's &lt;code&gt;format&lt;/code&gt;, then validated. On failure it retries with the validation error. Low-confidence extractions go to a "🔎 Needs a look" list instead of being silently wrong.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A lesson from the local models:&lt;/strong&gt; prompt-only email formatting was ignored (one-line bodies, greeting the wrong company). Moving layout into deterministic post-processing fixed all of it. Small models are great at &lt;em&gt;content&lt;/em&gt;; let code handle &lt;em&gt;form&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Eval.&lt;/strong&gt; &lt;code&gt;sloth eval&lt;/code&gt; runs the full pipeline on fake fixtures: receipt photos, PDF invoices, order emails, an mbox, a 13-month bank CSV, and a meme that must be ignored.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Score&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Field accuracy (merchant / amount / date)&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;96.7%&lt;/strong&gt; (29/30)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Deadline accuracy (type + date ±1 day)&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;100%&lt;/strong&gt; (14/14)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Subscription recall / flag accuracy&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100% / 100%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;False positives&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;0&lt;/strong&gt; (the meme is correctly ignored)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Time&lt;/td&gt;
&lt;td&gt;~3 minutes for the whole batch, fully local (M3 Max, 48 GB)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Stack:&lt;/strong&gt; TypeScript, Node 24 (built-in &lt;code&gt;node:sqlite&lt;/code&gt;), MCP TypeScript SDK, Ollama JS, zod, imapflow, Express 5 + SSE (a live agent log), React 19 + Vite + Tailwind v4, Vitest (39 tests).&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;For &lt;em&gt;this&lt;/em&gt; project it isn't a nice-to-have. It's the only way the project can exist.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Privacy.&lt;/strong&gt; To find a forgotten trial you need someone's bank statement, receipts and order emails: their whole financial life. With a closed API, that's a nightly upload of your bank CSV to someone else's server. With open weights it never leaves the laptop. &lt;code&gt;sloth doctor&lt;/code&gt; even fails if Ollama isn't on localhost, and the dashboard API only listens on 127.0.0.1.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Works offline.&lt;/strong&gt; Nothing in the pipeline needs the internet; try it with Wi-Fi off. A 2 AM job shouldn't depend on a connection or an API being up.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;$0 per run, forever.&lt;/strong&gt; It runs every night. A closed API would charge per token, every night, to save me money. Ironic.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Swappable.&lt;/strong&gt; Each agent's model is one line in a config. I switch between a 35B model and an 8B "lite" profile, and a &lt;code&gt;single&lt;/code&gt; profile uses one multimodal model for everything. No vendor lock-in, and no deprecation can break my nightly job.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tunable.&lt;/strong&gt; There's a &lt;code&gt;Modelfile&lt;/code&gt; that bakes in the Reader prompt, and because the weights are open you could fine-tune Reader on your own receipts.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No OAuth app for your inbox.&lt;/strong&gt; Mail sync is plain read-only IMAP to your own provider. No cloud project, no client secret, no third party reading your mail.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Open standards.&lt;/strong&gt; Because of MCP, the tools (ledger, docs, calendar, mail) work with any other MCP client, and the agents use the official filesystem server instead of me reinventing file access.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Where open worked &lt;em&gt;better&lt;/em&gt; than closed:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;JSON-schema-constrained decoding through Ollama's &lt;code&gt;format&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;keep_alive&lt;/code&gt; to keep models warm across a batch.&lt;/li&gt;
&lt;li&gt;The freedom to run a tiny vision model just for photos while a bigger one does the reasoning, on hardware I already own.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  My Agent Session
&lt;/h2&gt;

&lt;p&gt;I built Refund Sloth with &lt;strong&gt;GitHub Copilot CLI&lt;/strong&gt; as my pair programmer. It helped with:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;planning the agent pipeline;&lt;/li&gt;
&lt;li&gt;writing the MCP servers, the tests and the eval;&lt;/li&gt;
&lt;li&gt;iterating on the UI with headless-Chrome screenshots;&lt;/li&gt;
&lt;li&gt;recording the demo video.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Every non-obvious decision (why rules do the math, why read-only IMAP instead of OAuth, why "Cancel now" is a link and not auto-cancel) is logged in &lt;a href="https://github.com/amangithub01/refund-sloth/blob/main/DECISIONS.md" rel="noopener noreferrer"&gt;DECISIONS.md&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Lazy Payoff
&lt;/h2&gt;

&lt;p&gt;On the demo data (a realistic year of transactions), Refund Sloth found:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;$2,424 at risk in the next 30 days:&lt;/strong&gt; a $1,299 laptop return closing in 4 days, $348 headphones in 6, $249 AirPods in 8, and a trial that bills in 2.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;$86.96 a month (~$1,043 a year) you can cut:&lt;/strong&gt; a gym nobody visits ($24.99), an unused AI tool ($20), the most expensive of four streaming services ($17.99), and the extra music and cloud plans ($11.99 each).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;10 ready-to-send emails&lt;/strong&gt; and &lt;strong&gt;15 calendar reminders&lt;/strong&gt;, none of which I had to write.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Doing this by hand (digging through emails, checking each store's return policy, reading a bank statement line by line, writing polite-but-firm emails) is easily 1–2 hours a month. Now it's about 30 seconds of clicking Approve. 🦥&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of GitHub Copilot:&lt;/strong&gt; built end to end with the GitHub Copilot CLI.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>ai</category>
      <category>opensource</category>
    </item>
    <item>
      <title>BillBox: a bill-to-spreadsheet tool that runs on your laptop with an open model</title>
      <dc:creator>Aman Kumar</dc:creator>
      <pubDate>Sun, 18 Oct 2026 15:11:33 +0000</pubDate>
      <link>https://dev.to/aman_k_/billbox-a-bill-to-spreadsheet-tool-that-runs-on-your-laptop-with-an-open-model-66c</link>
      <guid>https://dev.to/aman_k_/billbox-a-bill-to-spreadsheet-tool-that-runs-on-your-laptop-with-an-open-model-66c</guid>
      <description>&lt;h2&gt;
  
  
  The problem
&lt;/h2&gt;

&lt;p&gt;Every month, small shop owners and freelancers sit down with a pile of bills and type them into a spreadsheet so their accountant can file GST. It's slow, and the bills contain GSTINs, bank details and customer names. Uploading them to a cloud OCR service is a bad trade for something this sensitive.&lt;/p&gt;

&lt;p&gt;So I built BillBox for my relative, who keeps a drawer full of paper bills and hands them to the accountant at the end of every quarter.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjvmq6ix0vk8jz3f512a9.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjvmq6ix0vk8jz3f512a9.png" alt="BillBox inbox showing invoices with Approved, Needs Review and Missing GSTIN status chips" width="800" height="391"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Drop in photos or PDFs of bills. A vision model running on your own laptop reads each one and extracts the vendor, GSTIN, invoice number, date and totals. A validation layer checks the math and flags anything suspicious. You fix flagged fields next to the original image, approve, and export a CSV your accountant can open in Excel.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Demo (sample data, no model runs here): &lt;a href="https://billbox-pi.vercel.app/" rel="noopener noreferrer"&gt;https://billbox-pi.vercel.app/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Code: &lt;a href="https://github.com/amangithub01/billbox" rel="noopener noreferrer"&gt;https://github.com/amangithub01/billbox&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Real app: runs locally with Ollama (setup in the README)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/vR_1ueRiwG0" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;h2&gt;
  
  
  Why open models mattered
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Privacy:&lt;/strong&gt; The bill never leaves the machine, because the model runs locally through Ollama.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost:&lt;/strong&gt; there's no per-page fee, so 500 bills cost the same as 5. That's what makes this viable for someone who earns little from it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Control:&lt;/strong&gt; the model is one config line (&lt;code&gt;qwen2.5vl:3b&lt;/code&gt; via Ollama). A bigger or fine-tuned model drops in without changing the app.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;My laptop has 8 GB RAM and no GPU, so I used the 3B model. That limits accuracy, and the next section is about what that meant in practice.&lt;/p&gt;

&lt;h2&gt;
  
  
  What went wrong (and what I did about it)
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The model is right-ish, not right.&lt;/strong&gt; My first test was an electricity bill. It returned valid JSON, but the date came back as &lt;code&gt;04-07-2018&lt;/code&gt; instead of &lt;code&gt;2018-07-04&lt;/code&gt;, and the total came back as a string, &lt;code&gt;"1031.00"&lt;/code&gt;, instead of a number. A spreadsheet can't use either. I added a normalization step that converts dates (day-first, the Indian format) and parses numbers like &lt;code&gt;"1,031.00"&lt;/code&gt; and &lt;code&gt;"Rs. 1031"&lt;/code&gt;, with unit tests for each case.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Small models make mistakes on digits, so I stopped trusting the output.&lt;/strong&gt; Instead of showing extracted values as facts, BillBox runs checks: GSTIN format and checksum, rate x quantity = amount, subtotal + taxes = total, and CGST/SGST vs IGST. A mismatch turns the field red with a plain-English message. On a bill with no GSTIN, BillBox flagged "Missing Vendor GSTIN" and sent it to Needs Review instead of guessing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Context limits.&lt;/strong&gt; Sending two images in one request crashed Ollama with a "request exceeds the available context size" error. The app now sends one image per request, resizes to 1280px, and sets a larger context window.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Not every bill is an invoice.&lt;/strong&gt; My electricity bill has no line items and no GSTIN, so the GST checks flagged errors that weren't errors. Now the checks are skipped when those fields don't exist.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;p&gt;Tested on 10 real bills (printed invoices, phone photos, one electricity bill), on an 8 GB Windows laptop with no GPU.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Field&lt;/th&gt;
&lt;th&gt;Correct&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Vendor&lt;/td&gt;
&lt;td&gt;10/10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Invoice number&lt;/td&gt;
&lt;td&gt;8/10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Date&lt;/td&gt;
&lt;td&gt;6/10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total&lt;/td&gt;
&lt;td&gt;8/10&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;About 55 seconds per bill. Validation flagged 4 wrong values that I would otherwise have missed. The worst results were on blurry phone photos and bills with no GSTIN printed on them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limitations
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;A 3B model needs human review. Treat the output as a draft.&lt;/li&gt;
&lt;li&gt;Roughly a minute per bill without a GPU.&lt;/li&gt;
&lt;li&gt;The hosted demo is simulated. The real extraction only runs locally.&lt;/li&gt;
&lt;li&gt;The CSV is meant for an accountant to open or import. It is not a ready-made Tally import file.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What's next
&lt;/h2&gt;

&lt;p&gt;Fine-tune on the corrections people make (every fix is a labeled example), add Tally-compatible export, and try a larger model on better hardware.&lt;/p&gt;

&lt;p&gt;Built with Qwen2.5-VL, Ollama, FastAPI, React and OpenCV.&lt;/p&gt;

</description>
      <category>hf26challenge</category>
      <category>ai</category>
      <category>opensource</category>
      <category>devchallenge</category>
    </item>
  </channel>
</rss>
