DEV Community: Amit

SlothBox: a local file butler that files your downloads, screenshots and receipts while you do nothing

Amit — Mon, 26 Oct 2026 04:55:38 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 3, Slow and Steady.

What I Built

SlothBox watches the folders you choose on a Windows PC, such as Downloads and Screenshots. When a new file lands, a local Gemma model reads it and says what it is: a receipt, a screenshot, a document, or something else. SlothBox then gives the file a clear name, moves it into a sorted folder, and, if it is a receipt, writes a row in a monthly ledger. A page you open on your phone over home Wi-Fi sends photos into the same pipeline. Every move can be undone from the dashboard or the tray.

Nothing leaves the PC. Gemma runs in Docker on your own GPU, the database is a local Postgres, and there is no account.

Who gets lazier

Anyone with a Downloads folder they have stopped looking at, a screenshot pile named Screenshot 2026-10-14 213055.png, or a wallet of paper receipts they mean to type into a spreadsheet one day.

Gemma never writes a file name. It returns fields, and plain code builds the name from a template:

Type	Template	Example
Receipt	`{date}_{Vendor}_{amount}{CUR}`	`2026-10-20_Jio_599INR.jpg`
Screenshot	`{file date}_{title}`	`2026-10-20_reddit-thread-on-dotnet-10.png`
Document	`{title}`	`dotnet-10-release-notes.pdf`
Other	original name, unchanged	`ChromeSetup.exe`

An installer keeps its name because renaming it would help nobody.

The rules that make it safe to leave running

Files that were already in a folder when you added it are never touched. "Tidy them now" is a separate click, and "Undo since…" puts everything back.
A file goes to Needs review instead of being guessed when Gemma's answer is not valid, when the type is unknown, when a receipt has no date or amount, or when the file could not be read (HEIC, scanned PDF, password-protected). Review is keyboard-only if you want it: a digit picks the category, Enter files it.
Every move is logged before it starts and finished through a temp file with a hash check, so a crash or a power cut leaves either the original or the copy, never neither.
Cloud-synced folders are copy-only. SlothBox never deletes from a folder that something else also owns.

Demo

SlothBox runs on your own PC, so there is no public deployment. Three screens from the build:

Today. The meter counts what was filed this week and turns it into minutes you did not spend. The two rows are test files: a synthetic text receipt filed under Receipts / Shopping, and a HEIC file that went to review and was filed by hand.

Needs review. SlothBox cannot read HEIC yet, so it asks instead of guessing. 1 to 9 picks a category, Enter files it, S skips, O opens the file.

Receipts. The ledger for October, with the total per currency and per category, and a CSV export. The one row is the synthetic receipt from the test above; Gemma read the date, amount and payment method and missed the vendor.

Code

AmitSahoo45 / slothbox

SlothBox

A local file butler for Windows. SlothBox watches the folders you choose, such as Downloads and Screenshots. A local AI model (Gemma, running in Docker on your own PC) reads each new file. SlothBox then gives the file a clear name and files it in a sorted folder. Receipts go into a monthly ledger, every move can be undone, and you can search your files by words or by meaning. A small upload page lets your phone send photos over your home Wi-Fi. Files never leave the PC.

What you need

Windows 10 or 11, 64-bit.
Docker Desktop with Linux containers.
An NVIDIA GPU with at least 8 GB of video memory is recommended. Without one, Gemma runs on the CPU and is much slower.
About 17 GB of free disk space for Docker: about 10 GB of images and 7 GB of…

View on GitHub

How I Built It

The open AI part

Two Gemma models through Ollama, both running in a Docker container on the PC:

Gemma 4 E4B (gemma4:e4b, 6.6 GB) reads each file. Images are sent as base64, PDFs and Office files as extracted text, and the model answers in a fixed JSON schema: type, category, title, and for receipts the vendor, date, amount, currency and payment method. Temperature 0, thinking off, one retry on a timeout or on malformed output. It fits entirely in 8 GB of VRAM, so the GPU runs one file at a time with nothing offloaded to RAM.
EmbeddingGemma (embeddinggemma, 768 dimensions) turns the text Gemma wrote about each file into a vector, on the CPU, so Gemma keeps the GPU to itself.

The model proposes; code decides. The category must be one of the user's own list, or it becomes Other. A currency that is not a three-letter code becomes null. An amount of ten billion or more becomes null, because the ledger column is numeric(12,2) and a misread would otherwise fail the insert. Control characters are stripped before anything reaches Postgres.

Storage and search: Postgres + pgvector

Everything that is not a file lives in one Postgres database: the queue, the action log, the receipts ledger, the categories and the search index. Search is one SQL query that merges the top 50 keyword matches (tsvector, websearch_to_tsquery) with the top 50 meaning matches (cosine distance on the EmbeddingGemma vector) using reciprocal rank fusion with k = 60. Each hit reports whether it matched by words, by meaning, or both, and the UI shows that label.

Two things in that query were not in any tutorial I read:

pgvector's default hnsw.ef_search is 40, which is below LIMIT 50, so without SET LOCAL hnsw.ef_search = 100 the meaning branch silently returned at most 40 rows.
With a filter such as "receipts only", the HNSW index picks its nearest rows first and applies the filter after, so a narrow filter could return no meaning matches at all. SET LOCAL hnsw.iterative_scan = strict_order fixes that.

The test for the second point is the most measured line in the repo. The index graph is built with Postgres's internal random generator, which no SQL setting can seed, so the same 20 seeded receipts come back as 20 matches about four runs in five and as few as three in the worst of 600 runs. The test now asserts what the spec can justify, at least one, and the spec records the numbers.

The stack

A .NET 10 tray app hosts an ASP.NET Core server on 127.0.0.1:5080, with the React dashboard built into the exe. A second listener on port 5081 serves only the phone page and its upload. Postgres and Ollama run from one compose file. The release is a single self-contained SlothBox.exe, so the PC that runs it needs no .NET install.

The engine has 768 tests and the UI has 139, all on synthetic files and a throwaway Postgres container, so no real file of mine is in the repo.

The bug I am glad a review found

Undo looked simple: the action log says the file went from A to B, so move it back. A review of the first safety batch found that a stale undo, one requested after the file had already been refiled somewhere else, could build a "delete the copy" step whose copy and original were the same path, which would have deleted the only copy of the file. Three guards went in: undo refuses unless the action is still the file's newest live one, the action log re-checks that under a row lock, and the mover refuses any delete whose copy and original are the same path. Each guard has a test that failed before it existed.

How the work was split

I used AI for most of the code. Claude (Opus 5.5, through Claude Code, on the laptop and in cloud sessions) wrote the engine and most of the UI from a design spec and a task plan, test first, one batch at a time, each batch ending with a report and a review. OpenAI's Codex (GPT-6 Astra) built the first four UI batches. My part was choosing the problem, writing the rules into the spec, deciding every open question the agents raised, merging the branches, running the checks, and running the app on my own machine against a real phone and real folders.

Why Does Open Innovation Matter?

Downloads folders are full of things nobody would upload on purpose: bank statements, ID scans, chat screenshots, receipts with a home address on them. A closed vision API would mean sending every one of those files to someone else's server, paying per file, and needing internet before a screenshot could be filed.

With open-weight models, the files never leave the PC. Once the two models are downloaded, a file costs nothing to process, and the whole thing works with Wi-Fi off, apart from the phone page, which needs the local network and nothing else.

Open also means the model is a setting. The spec has a day-one rule: if gemma4:e4b does not fit the GPU or takes more than ten seconds per file, switch to the quantised build or to gemma4:e2b. That is one line in settings.json and a pull in Ollama, not a vendor negotiation. The same goes for the embedding model, with a re-embed of the index.

And it means the limits can be measured and published. The receipt fields Gemma gets wrong, the search matches the index misses, the seconds each file takes: all of that is visible, repeatable, and free to re-run.

The Lazy Payoff

The dashboard has a meter. The formula is shown in the UI under "How we count" and it is an estimate, stated here plainly: a receipt counts as 30 seconds you did not spend typing it into a spreadsheet, a screenshot or a document as 15 seconds you did not spend renaming and moving it, anything else as 5 seconds. The per-type seconds can be changed in Settings.

I did not get SlothBox onto my real Downloads folder before the deadline. The week went into the build, and the plan's "run it on real folders from day one" step is the one that slipped. So the numbers below come from a test folder and the phone page, and they are small:

Files filed: 3. One photo uploaded from the phone, one synthetic text receipt, one HEIC filed from review.
Receipts logged: 1, ₹120.00. Gemma read the date, the amount and "paid in cash" correctly and missed the vendor, so the file is 2026-10-26_120INR.txt instead of 2026-10-26_SampleStore_120INR.txt.
Duplicates caught: 0.
Minutes saved by the formula: under one.
Seconds per file: 28 s for the first file after a start, while Gemma loads into the GPU; about 2 s per call once it is warm. The spec's target of under ten seconds per file holds only once the model is loaded.

That is one receipt, not an accuracy figure. The honest version is: the pipeline works end to end, and the measurement of how often Gemma gets a receipt right is the first thing to do after the deadline.

Prize Categories

Best Use of Gemma. Gemma 4 E4B reads every file; EmbeddingGemma indexes it.

Credits

Models: Gemma 4 and EmbeddingGemma by Google, under the Gemma Terms of Use, served by Ollama.
Database: PostgreSQL with pgvector.
.NET libraries: Npgsql and EF Core, PdfPig, SkiaSharp, OllamaSharp, Testcontainers.
UI: React, Vite, Mock Service Worker.
The hybrid search is the same idea I used in my earlier rag-semantic-search project; the SlothBox code was written fresh.

If you have a Downloads folder you have given up on, point SlothBox at a copy of it and tell me what it got wrong. Open an issue on the repo.

BibLocal: find every photo of a runner by bib number, offline, on a laptop CPU

Amit — Mon, 19 Oct 2026 04:46:15 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 2, Side Hustle.

What I Built

BibLocal reads the bib numbers in a folder of race photos and writes them into each file as keywords. After that, a photographer can click a bib number in their photo manager and see every photo of that runner.

It runs on a laptop CPU with Wi-Fi off. Nothing is uploaded. There is no account, no API key and no GPU.

Who it's for

Race photographers. One 5K or trail race can mean thousands of photos, and every runner wants to find theirs. The usual way to make that possible is to open each photo, zoom in on the bib and type the number.

Photographers have complained about this for years. On the dgrin forum, a ski-race photographer shooting 2,000 to 3,000 images per race put it at "maybe 90 minutes to keyword a race." In 2024, someone on the Adobe Community asked whether any Lightroom Classic plugin could recognise race bib numbers. The Community Expert's answer was "No."

I didn't start with this idea. I started with filters. While researching ideas with Claude, I threw out the first batch: one needed a GPU the user wouldn't have, and another was something a free chatbot already does. The next round had to run on an ordinary CPU, beat a free chatbot, depend on open AI, and help people anywhere, not just in my city. When I read about BibLocal, I knew it was solving something real.

Who would pay for it

Photographers already pay for this. Manual tagging services charge about 2 cents to 5 cents per photo. For a 3,000-photo race, that is $60 to $150, every race. Cloud tools charge credits or a subscription, and the photos have to be uploaded first.

BibLocal is free and MIT licensed. [CHECK: the paid part. Suggestion: a signed one-click Windows build and setup help, sold per photographer instead of per photo.]

How it works

biblocal scan reads every JPEG in a folder and proposes bib numbers. It changes no photo.
Photos it isn't sure about go to a local review page. You confirm or type the bib, press Enter, and download corrections.csv.
biblocal apply with --write embed writes the confirmed bibs into the JPEGs as IPTC and XMP keywords, using exiftool. Every original is kept as a backup.

Demo

About a minute, no audio. It runs on 29 real race photos from friends who run marathons; they agreed to me using them. The scan is shown at 5x speed. The real scan time was 51.42 seconds. I didn't get a clean recording of the apply step, so the video shows a text card with the result of the real run instead: 25 of 26 photos written.

Code

AmitSahoo45 / biblocal

BibLocal

What it does

BibLocal reads runners' bib numbers in JPEG race photos on a CPU, offline Default scans preserve photos and write a CSV and local review page. Opt-in writes add bib keywords.

Quick start

Tested on Windows 11 with Python 3.13.7. Install Python 3.13 and this checkout on D:. Run in PowerShell from the checkout:

$env:PIP_CACHE_DIR = "D:\.cache\pip"
New-Item -ItemType Directory -Force .tmp | Out-Null
$env:TEMP = (Resolve-Path .tmp).Path
$env:TMP = $env:TEMP
py -3.13 -m venv .venv
.venv/Scripts/python.exe -m pip install .
.venv/Scripts/biblocal.exe scan "D:\Photos\Race" --start-list "D:\Photos\start.csv"

Setup needs package access or prepared local wheels. It installs CPU ONNX Runtime and RapidOCR's bundled models No CUDA, GPU runtime, account or API key is needed. This is a CLI, with no…

View on GitHub

How I Built It

The open AI part

The OCR is PaddleOCR's PP-OCRv6 small detection and recognition models (Apache-2.0), run through RapidOCR 3.10.0 on the CPU build of ONNX Runtime. The two model files are about 31 MB together and ship inside the RapidOCR package. BibLocal checks their SHA-256 hashes before using them and never downloads a model.

Each photo is decoded at reduced size with Pillow's JPEG draft mode, turned upright from its EXIF orientation, resized to a long side of 2,048 px and passed to the OCR.

The OCR proposes, plain code decides

I didn't want a model deciding what gets written into someone's files. So the OCR only produces candidates, and ordinary Python checks each one:

Digits only, with a sensible length.
Confidence of at least 0.90 to tag. Between 0.50 and 0.90, the photo goes to review.
On the race's start list, when one is supplied.
A text box at least 2% of the photo's height, so tiny numbers in the background don't count.
Not a number that shows up across too many photos. Those are usually banners or signs, not bibs.

A photo that fails these checks goes to review or gets no tag. A suggestion is never written as a tag.

How the work was split

I used AI for most of the code, so here is exactly who did what. Claude (Opus 5.5) did the research, the design spec and the task plan, and later edited the demo video. OpenAI Codex, running GPT-6 Astra, wrote most of the code from that plan, test first, one task per commit. My part was choosing the problem, setting the rules, making the call when the numbers disagreed, checking labels and results, running it on real photos, verifying the tags in digiKam and recording the demo.

The rules were written down before any code, in the repo's CLAUDE.md. CPU only, even though my laptop has a GPU. No network calls at runtime. Never write to a photo unless asked. When in doubt, send it to review. Permissive licences only, and every model file recorded with its source, size, checksum and licence. The final suite has 134 tests, all on synthetic images, so no real person's photo is in the repo.

The hardest bug: IPTC

Writing a keyword into a JPEG sounds trivial. It wasn't.

Older photos often carry IPTC metadata encoded as Latin-1. Telling exiftool to treat it as UTF-8 reinterpreted those bytes and corrupted the text; a test with "café" and "El niño" reproduced it. Copying the old IPTC and editing keywords in one command duplicated keywords. Worst of all, exiftool can cut an IPTC keyword longer than 64 bytes and still exit with status 0, leaving only a warning.

The fix was two separate exiftool commands per photo, run on a staged copy, with any warning treated as a failure. If the write isn't clean, the original stays exactly as it was.

That rule fired on the real apply run. One photo triggered an exiftool warning about its HDR gain-map data [CHECK wording], so BibLocal skipped it and wrote the other 25.

The 10-out-of-30 moment

I tuned the rules on 100 photos from RBNR, a public research dataset of race photos with labelled bibs. Then I asked friends who run marathons for their race photos, and they sent me 30.

Before the scan, I asked myself: if it fails on real photos, has the project failed?

The first scan tagged 10 of them.

The cause was a rule tuned on the dataset. The minimum bib height was 4% of the photo, because that worked on RBNR. In my friends' photos, many bibs are smaller in the frame, and the rule threw them away. At 2%, the scan in the demo gave 23 tagged, 3 sent to review, 3 with no readable bib, and 0 errors. (That run uses 29 photos. I left one out of the video.)

That change has a cost, and the dataset shows it:

Minimum bib height	Photos with a correct tag	Wrong tags, realistic start list
4%	80%	7.76% (9 of 116)
2%	87%	19.08% (29 of 152)

"Realistic start list" here means every number between the lowest and highest bib at each event, which is close to what a real race list looks like. So at 2%, about one in five tags it accepts on that dataset is wrong. My target was under 3%. It misses that by a long way.

I kept 2% as the default because a tool that misses most bibs on real photos is no use to anyone, and a wrong tag is visible in the photo manager and can be removed. [CHECK: your reason] If you would rather miss bibs than tag wrong ones, --min-bib-height 0.04 brings back the old behaviour.

One more caveat about the demo: its start list contains only the 30 bibs that appear in those photos. That is the best case. A real race list can have thousands of numbers, which gives a misread more chances to match one.

What it doesn't do yet

The wrong-tag rate above is the first thing to fix. The dataset numbers are also in-sample: 100 photos, no held-out set.
I checked the keywords in digiKam 9.1.0. Lightroom Classic is untested.
Speed comes from one laptop with an Intel Core i7-13650HX. On mains power, a 24 MP test image took about 1.5 seconds, roughly 25 minutes per 1,000 photos (an estimate, not a measured batch). On battery saver, it took 12 to 14 seconds per photo.
It's a command-line tool, tested on Windows 11 only, and it reads JPEGs only.
Blurred, covered or angled bibs get missed. Someone still has to review.

Why Does Open Innovation Matter?

Race photos are photos of the public, often including children. With a closed vision API, every one of them would be uploaded to someone else's server and paid for per image, and the tool would need internet at the finish line.

With an open model, the photos never leave the laptop. You can check that yourself by running the scan with Wi-Fi off; I did. Once it's installed, a photo costs nothing to process.

Open also let me measure it properly. The model files are fixed and checksummed, so the OCR gives the same output every time. The calibration script caches the OCR output for the 100 dataset photos and replays the rules under 18 combinations of settings, which is how the thresholds were chosen. With a closed API, the model can change under you, and every re-run is a bill.

If a better PP-OCR comes out, or someone fine-tunes one on bib fonts, swapping it in means changing a file and re-running the calibration.

Open did not make the model good at bibs. It gave me a model I could run as often as I liked and actually measure. That's how I know about the 19%, and I would rather publish that number than hide it.

Learning Dad's Garden: From 4 Minutes of Phone Video to a Printed Watering Sheet, Offline

Amit — Sun, 11 Oct 2026 19:08:54 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

My dad has looked after our pot plants for years. On Sunday morning he filmed them on his phone and explained each pot as he went.

At the bush with tag 12, he said: "Give regular amount of water, the usual amount."

He knows exactly how much that is. I didn't. So I built a tool that finds every line like that and asks him.

It's called Learning Dad's Garden. It turns his walk-through into a printed checklist I can take outside. Each pot gets a row, with a tick box for each day it needs something. Gemma 4 does the reading, on my laptop. Before anything prints, Dad answers the questions a beginner would ask.

What I Built

I'm taking up Dad's hobby, and he knows what every pot needs. Some of his instructions are exact. Plant with tag 4 gets "200 to 250 ml of water daily". For the pot with tag 9 he said "one cup of water or one and a half cup of water to be accurate, the green cup we have", which is exact, as long as nobody moves the green cup. Others only make sense if you already know the garden: "the usual amount", "water it regularly", "this plantation". To him, those are complete instructions.

Learning Dad's Garden turns a walk-through like that into something I can follow:

Dad films each part of the garden and talks, the way he would explain it to me in person.
"Learning Dad's Garden" turns his speech into text and splits it into instructions. Then it checks that every amount and time came from his words.
A gap check lists everything I couldn't act on. Dad answers on a local web page that shows the video frame and plays his voice from that moment.
The sheet prints. Each row has the tags, what to do, how much and when, a box for each day it's due, and Dad's exact words underneath.
I take the sheet outside, water, and tick. Each evening Dad checks the pots and marks every row: done right, missed or done wrong. My dad is grading my homework again.

The screen time comes first but it stays small. The routine lives on paper. This week I walk from pot to pot with the sheet and a pencil, and the laptop stays shut.

It works for anyone learning a garden from someone who knows it by feel. The command line calls them the owner and the helper. In our house, that's Papa and me.

The top of our sheet. Tag 12's "usual amount" is now half a glass of water, in Dad's typed words. What he said on Sunday sits under the row.

Demo

Code

GitHub URL

The licence is MIT.

It's built on faster-whisper, Ollama, Gemma 4, PyAV, Pillow, Jinja2 and headless Chromium for the PDF.

How I Built It

The model has one narrow job, and plain Python checks its work at every step. Pipeline :

Step	Done by
Speech to text, with the time of every word	faster-whisper `small.en`, on the CPU
Split into instructions	Gemma 4 E4B through Ollama, held to a JSON schema
Grounding check	Python
Days on the sheet	Python rules
Gap check	Gemma's flags, a phrase list, structure rules
Review	A local page: standard-library server, plain JavaScript
Print	Jinja template, then headless Chromium

What Gemma is allowed to do

Gemma gets the transcript of one part of the garden and a short list of rules. The first rule sets the boundary:

You are not a gardening expert. Never add advice, amounts, times or conditions that the speaker did not say.
(Well, neither am I, which is the whole reason this project exists.)

Ollama's structured output holds Gemma to a JSON schema. For each instruction, Gemma returns the exact quote, the tag numbers, the kind (do, don't, do only if, or a note), a short command, the amount, the timing and the condition. It also lists any words a helper couldn't act on. An example:

{
  "quote": "water this daily in the morning or in the evening anytime when you are free give regular amount of water, the usual amount",
  "pots_said": "",
  "pots": [],
  "kind": "do",
  "what": "Water",
  "amount": "regular amount of water, the usual amount",
  "when": "daily in the morning or in the evening anytime when you are free",
  "condition": "",
  "vague": [
    {"kind": "which_pot", "phrase": "this"},
    {"kind": "how_much", "phrase": "regular amount of water, the usual amount"}
  ]
}

Gemma flagged both gaps on its own. It couldn't tell which pot "this" was, or how much "the usual amount" is. Both became questions for Dad.

I kept Gemma's thinking on, which is its default, with temperature 0 and an 8,192-token context. Gemma 4 E4B can take audio as input, but Whisper still does the listening. The tool needs two things Whisper gives for every word. The time lets the review page play the right clip. The confidence lets the sheet mark words Whisper wasn't sure of. I also give Whisper a list of words from Indian gardens, like tulsi, money plant, curry leaves, lota, etc. so it expects them.

Every value has to be in Dad's words

A model that restructures text will sometimes try to help. So nothing Gemma returns goes straight onto the sheet. First, the code finds each quote in the transcript, word for word or close to it, with a similarity of at least 0.8. Then it checks every amount, time and condition. A value must be copied from the instruction's own words, or from elsewhere in the same video. Every number in it must appear there too. Anything else is deleted.

def ground_field(value: str, quote: str, transcript: str) -> str | None:
    """Where a field value came from.

    "said": copied from the instruction's own words.
    "inferred": copied from elsewhere in the same note (for example "do the
    same for 26"); the owner is asked to confirm it.
    None: not found; the value must be dropped.
    """
    if not value or not value.strip():
        return None
    if is_extractive(value, quote) and numbers_ok(value, quote):
        return "said"
    if is_extractive(value, transcript) and numbers_ok(value, transcript):
        return "inferred"
    return None

"Inferred" covers lines like "do the same for 26". The value is real but comes from another sentence, so the tool asks Dad to confirm it.

Days come from rules

The model never decides days. Plain rules turn the timing words into days on the sheet. "Daily" means every day. "Every alternate day", "one day gap" and "every 2 days" all mean days 1, 3, 5 and 7 on a 7-day sheet. That makes 11 rows on odd days and 5 on even days. A striped box means nothing to do that day.

The questions

The gap check rebuilds its questions from the instructions whenever anything changes, and it never stores a question as the truth. An answer changes the instruction. "It's fine as is" dismisses that one question. The questions come from five places:

Gemma's own flags, like the two above.
A phrase list of things people say instead of numbers. Three of its lines:

  ("how_much", r"\b(?:as usual|the usual(?: amount| way| quantity)?|like (?:usual|always|i do|before)|the same amount)\b"),
  ("how_often", r"\b(?:regularly|sometimes|occasionally|now and then|from time to time|often|frequently"
                r"|once in a while|on and off)\b"),
  ("when_to_act", r"\bdepend(?:s|ing)?(?: on| upon)?(?: how| what| whether| if| the)(?: \w+){0,4}"),

Structure rules: a watering row with no amount, two amounts joined by "or" (tag 9's "100 to 200 ml or 250 ml"), a command that is only a verb and an amount (tag 11's "Keep around 200 ml"), and a Do NOT line that also has a timing.
Coverage: tags that no instruction names.
Whisper: a word it heard with less than 5% confidence gets a question if it lands in a field on the sheet. In Dad's quotes it gets a "(?)", as in "amount brown (?) should be not more than 200 ml".

Every question carries its moment in the video. The page shows the frame and plays Dad's voice from the first word of the instruction to the last. He doesn't have to remember what "this" meant, because he sees what the camera saw. For pot with tag 12 he typed "This is tag number 12", then "that is usually 1/2 glass of water".

The tag 6 question, on a copy of the project without our videos, so the frame and audio player are missing. The green line repeats the previous answer back.

For the record, Plant (Tag #12) usual amount is half a glass of water. It took a 19-second video, a local AI model and one question to find that out. I could also have just asked him.

Why Does Open Innovation Matter?

The videos show our front door, our pots and my dad's voice. With Gemma and Whisper running on my laptop, none of it leaves the house, and after the first model download a full run works with the Wi-Fi off. Each run costs nothing, so I could re-run it as often as I liked. Anyone taking over a garden can run it too. It needs Docker and, for speed, an NVIDIA GPU, with Gemma 4 E2B as the smaller fallback.

Prize Categories

Best Use of Gemma. Gemma 4 E4B does all the reading. It splits Dad's speech into instructions, flags the words a beginner can't act on and, as an option, reads tag numbers in video frames. It runs locally through Ollama.

BreakMyQuery: find the row that breaks your SQL

Amit — Sun, 04 Oct 2026 14:55:47 +0000

BreakMyQuery: find the row that breaks your SQL

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

I built BreakMyQuery for a colleague(also a friend🫠) who is a new graduate hire and is struggling with SQL joins or let's say he's scared of SQL joins.

A query can pass the sample data while quietly dropping a customer who has no orders. For someone learning joins, seeing the missing customer gives them a concrete place to start investigating.

BreakMyQuery offers eight SQLite exercises covering joins, NULLs, aggregation, ties, arithmetic, dates, and anti-joins. The learner writes a query, runs it, inspects any counterexample, and tries a repair. Each counterexample shows the breaking dataset, the learner's result, and the expected result. My traps saves their mistakes so they can revisit a query through Retry.

I have demonstrated the app to my friend(or colleague). The goal is to help them connect the query they wrote to the rows it keeps, loses, or duplicates.

Demo

Try BreakMyQuery.

The first exercise asks for every customer's name and their number of paid orders, including customers with zero paid orders.

That one row exposes the problem. The learner can trace why Alice disappears and change their query. The app keeps the reference SQL out of the interface.

The hosted verification record documents the tested counterexample, accepted-query, error, history, and Retry paths.

Code

Source code, exercises, tests, and setup instructions.

The application code uses the MIT license. The model weights have their own licenses.

How I Built It

The stack is Python, Streamlit, SQLite, and Pydantic. The hosted demo uses the open-weight Gemma 4 26B A4B model through Google's Gemini API.

The app first checks the sample. If the results match, it searches for a breaking case:

Propose. The model receives the schema, exercise question, learner query, and reference query. It proposes small datasets intended to make the queries disagree.
Verify. The app validates the proposed data and executes both queries in SQLite. It compares results while preserving duplicate-row counts.
Shrink. It removes rows while preserving the mismatch and database constraints.
Practice. The learner sees the remaining data and differing results, then edits their own query.

Seeded random fuzzing generates additional datasets and provides a fallback when the model returns no usable proposal or the API is unavailable. The hunter does not use the exercise's authored trap fixtures.

What the evaluations showed

I evaluated model proposals and fuzzing separately against 14 deliberately wrong queries and eight equivalent alternatives.

Method	Wrong queries exposed	False positives on alternatives
Gemma 4 via API	14/14	0/8
Using Local Llama 3.2	9/14	0/8
Seeded random fuzzing	14/14	0/8

The Gemma catalog evaluation used live API calls from the local checkout; the deployed app was checked separately.

Gemma's figures combine the primary run with one explicit follow-up after an unusable response. Covering the 22 cases took 23 model attempts. The Gemma report preserves that failed call; the Llama report records the separate local trial.

On this catalog, random testing caught five mistakes Llama missed. Keeping both methods gives the app another chance to find a breaking case.

I also found a teaching problem: Llama could produce a valid counterexample and still explain its cause incorrectly. I disabled generated hints and mistake labels in the demo. Gemma's teaching quality remains unevaluated. The tables and query results give the learner evidence they can inspect.

These measurements cover a small authored catalog. A passing run means no counterexample was found within the search budget. It does not prove the query correct.

Why Does Open Innovation Matter?

Open weights give this project a local deployment option. Once the dependencies, runtime, and model are installed, the Ollama workflow runs inference and SQLite on the learner's computer without a hosted inference API.

The public demo offers an easier starting point: my friend can open a link without installing a model. That convenience has a different data path. Submitted SQL goes to Google, and SQLite verification runs on the app server. The interface discloses this.

I could evaluate local Llama and Gemma through the API using the same SQLite checker. Another developer can inspect that checker, swap the model, and repeat the evaluation. The model's role is explicit: it proposes cases, and SQLite checks whether they demonstrate a mistake.

The counterexample approach was inspired by XData at IIT Bombay. BreakMyQuery applies that idea to a small practice tool for my friend: find a concrete case, inspect the rows, and work out the repair.

Prize Categories

Best Use of Gemma: Gemma 4 proposes the breaking datasets used by the hosted demo. SQLite verifies each accepted counterexample before the learner sees it.

How AI text watermarking works

Amit — Sun, 27 Sep 2026 16:58:01 +0000

An AI model can leave a watermark in the words it chooses. The text looks ordinary, and the mark can survive being copied into Notepad because it is carried by the token sequence. A detector with the matching key can check for that pattern later.

Anthropic explained its approach in August 2026 in "How Claude's text watermark works". It uses a version of SynthID-Text, which Google DeepMind published in Nature in 2024. Anthropic says it is introducing watermarking to meet the EU AI Act's transparency requirements.

I wanted to understand how ordinary word choices could carry a signature, and what a detector finding that signature would actually tell us. I followed the choice of one token through the process, then tested a small implementation with GPT-2 in Google Colab.

Starting with one word

Let's say you upload a photo of a cat to Claude and ask, "What is the cat in this image doing?" Claude starts its answer with "The cat" and now has to pick the next token.

A language model writes one token at a time. A token can be a word, part of a word or punctuation. GPT-2's tokenizer for example, has roughly 50,000 tokens. The model assigns probabilities to possible next tokens. For a simplified example, assume only these three options have non-zero probabilities:

sits: 50%
sat: 30%
lies: 20%

These are invented probabilities and not actual measurements. We will treat each word as one token throughout the example. Without a watermark, sampling from these probabilities would pick "sits" about half of the time. The chosen token joins the sentence, and model calculates probabilities for the next token. The watermark changes how the next token is selected. Model's weights remain unchanged. For the same input, it still produces same initial probabilities. Watermarking just influences the selection.

Without the watermark, a model might write "The cat sits on the mat." With the watermark, it might write "The cat sat on the rug" instead. These are just illustrative outcomes. The choices happen as the text is generated.

A secret key, combined with the preceding tokens, determines the watermark scores for each possible next token. Changing the preceding tokens can change the scores. With same key, settings and token context, calculation gives the same scores every time. That repeatability makes detection possible later.

The tournament

SynthID-Text uses a method called tournament sampling to choose the next token. We can follow it through an example with four candidates and two rounds.
Suppose a model has written "The cat" and assigns these probabilities to the next token:

Token	Probability
sits	50%
sat	30%
lies	20%

First, we draw four candidates using those probabilities. Each draw can pick any of three tokens, including one already picked. One possible result is:
sits, sat, sits, lies

"sits" appears twice in this draw. It's 50% probability makes it more likely to appear, but that does not guarantee two places.
Next, watermarking algorithm uses key and preceding tokens to calculate a score of 0 or 1 for each candidate. Each round has its own scoring function. Within a round, repeated copies of the same token receive the same score. Suppose scores produce these matches:

Round	Match	Winner
1	sits (0) versus sat (1)	sat
1	sits (0) versus lies (1)	lies
2	sat (1) versus lies (0)	sat

The higher score wins each match. Ties are broken randomly. A token's score can differ between rounds, which is why "lies" scores 1 in the first round and 0 in the second. The winning token is "sat", so text now reads "The cat sat". Generation continues from there.

Although "sits" had the highest original probability, both of its copies lost in this tournament. Another draw could produce a different winner. The model's probabilities determine how likely candidates are to enter, and watermark scores determine which candidates advance. This is method described in SynthID-Text paper.

Picking "sat" once provides little evidence of a watermark. An unwatermarked model could pick it too. Across many tokens watermarked text tends to have higher watermark scores than expected by chance. The detector measures that tendency. The bracket helps explain the method. The implementation discussed later calculates the equivalent winning probabilities directly.

Does watermarking affect writing quality??? 🤔

Drawing candidates from the model's probabilities keeps likely continuations well represented. That alone does not guarantee good writing. SynthID-Text has a quality-preserving configuration and another that produces a stronger watermark at a cost to quality. Google tested the quality-preserving configuration on nearly 20 million Gemini responses. The paper reports no statistically significant difference in thumbs-up or thumbs-down ratings between watermarked and unwatermarked responses. When the model has very little choice about the next token, there is less room to leave a watermark. This makes detection harder in passages where accurate wording leaves few alternatives.

How a detector checks without the prompt

The detector uses text, secret key and same watermark settings used during generation. It also needs the matching tokenizer to split the text into tokens. For each token, it takes the preceding tokens and calculates that token's watermark scores again. The calculation is repeatable: the same inputs produce the same scores.

This works without the prompt because the preceding tokens are already in passage. With a four-token context, detector skips scoring the first four tokens. From the fifth token onward, it has the context it needs. A simple detector averages recovered scores across the eligible tokens. It skips repeated contexts so that a repeated phrase does not count as fresh evidence. SynthID-Text also supports more sophisticated detectors.

For text written without this watermark, average is expected to be around 0.5 because the scoring functions assign roughly equal numbers of zeros and ones. Watermarked text tends to score higher because tournament favours tokens with higher scores. An average above 0.5 is not enough on its own to establish a watermark. Short passages can score high by chance. Longer passages usually provide more evidence, provided enough of the watermark survives.

Detectors use thresholds calibrated against unwatermarked text to control false positives. For a simple average-based approach, the threshold also needs to account for text length. Editing can weaken the signal because it changes the inputs to the scoring functions. With a five-token window, replacing one token affects the score inputs for that position and the next four. Replacing a whole word may affect more tokens, depending on how it is tokenized. Small edits can leave much of the original signal intact. Extensive rewriting can make the watermark undetectable.

How the code implements the tournament

A tournament with 30 rounds would require 2³⁰ candidates for each token, which is over a billion. The implementation calculates the equivalent winning probabilities directly.

for i in range(depth):
    g_values_at_depth = g_values[:, :, i]
    g_mass_at_depth = (g_values_at_depth * probs).sum(axis=1, keepdims=True)
    probs = probs * (1 + g_values_at_depth - g_mass_at_depth)

g_values contains the watermark scores, and depth is the number of rounds. Each round adjusts the probabilities to favour tokens that scored 1. The adjustment depends on how much probability those tokens already hold. Using our cat example, the probabilities change as follows. In round 1, "sat" and "lies" score 1. In round 2, only "sat" scores 1; "sits" and "lies" both score 0.

Token	Original probability	After round 1	After round 2
sits	50%	25%	13.75%
sat	30%	45%	69.75%
lies	20%	30%	16.5%

The 69.75% comes from the chance that at least one finalist is "sat". After round 1, each finalist has a 45% chance of being "sat", so each has a 55% chance of being something else. The chance that neither is "sat" is 0.55 × 0.55 = 0.3025, or 30.25%. Since "sat" beats the other tokens in round 2, its chance of winning is 100% - 30.25% = 69.75%.

The implementation calculates these winning probabilities and makes one random choice from them. This gives each token the same chance of selection as running the tournament. "sat" is the most likely choice, but the other tokens still have a combined 30.25% chance.

Testing with GPT-2

I used GPT-2 in Google Colab to generate text with and without a watermark. The setup used 30 key values for 30 watermarking rounds, a five-token window, top_k=40 and temperature=0.7. Each response could contain up to 300 new tokens.

For each of five prompts, I generated one watermarked response and one plain response. I then took the first 20 words of the watermarked response. Separately, I made an edited version of the full watermarked response by replacing one word in four, starting with the first, with a random choice from "the", "a", "very" and "also". These replacements test the effect of disruptive edits rather than natural paraphrasing. I also scored an F1 text sample.

The detector averaged the watermark scores after excluding the opening tokens, repeated contexts and end-of-text padding. It produced a raw average score, not a probability that the text was AI-generated.

The first cell loads GPT-2 and sets the watermark configuration:

import torch, random
from transformers import (AutoTokenizer, AutoModelForCausalLM, SynthIDTextWatermarkingConfig, SynthIDTextWatermarkLogitsProcessor)

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
tok = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2").to(DEVICE)

CFG = dict(ngram_len=5, keys=[654,400,836,123,340,443,597,160,57,29,590,639,13,715,468, 990,966,226,324,585,118,504,421,521,129,669,732,225,90,960],sampling_table_size=2**16, sampling_table_seed=0,context_history_size=1024)
wm_config = SynthIDTextWatermarkingConfig(**CFG)
lp = SynthIDTextWatermarkLogitsProcessor(**CFG, device=DEVICE)

The second generates a response with or without the watermark:

def gen(prompt, watermark, n=300):
    inp = tok(prompt, return_tensors="pt").to(DEVICE)
    out = model.generate(**inp, do_sample=True, top_k=40, temperature=0.7,max_new_tokens=n, pad_token_id=tok.eos_token_id, watermarking_config=wm_config if watermark else None)
    return tok.decode(out[0, inp.input_ids.shape[1]:], skip_special_tokens=True)

The third calculates the average watermark score:

def score(text):
    ids = tok(text, return_tensors="pt").input_ids.to(DEVICE)
    g = lp.compute_g_values(input_ids=ids).float()  
    eos = lp.compute_eos_token_mask(input_ids=ids, eos_token_id=tok.eos_token_id)[:, CFG["ngram_len"]-1:]
    rep = lp.compute_context_repetition_mask(input_ids=ids)
    m = (eos * rep).float().unsqueeze(-1)
    return ((g * m).sum() / (m.sum() * g.shape[-1])).item()

The fourth runs the comparisons:

prompts = ["Write about how coffee is grown and exported.", "Explain how trains work.",  "Descrieb a day at the beach.", "Tell the history of the bicycle.", "Explain how rain forms."]
human = """George Russell won the Azerbaijan Grand Prix in Baku on Saturday, 26 September, converting pole position into his third victory of the season. It was not a quiet afternoon. The race had two safety car periods, six retirements, and a finish so close that Russell crossed the line only a tenth of a second ahead of Max Verstappen. Isack Hadjar completed the podium in third place.

The first safety car came out after Alex Albon made an error and binned his Williams. The second followed by Franco Colapinto torpedo'ing himself in turn 1, which also ended the races of his teammate Pierre Gasly and Lando Norris. Both Aston Martins retired with mechanical problems. Oscar Piastri led for much of the race, but a lock-up cost him badly and he finished fourteenth. Charles Leclerc took fourth and Lewis Hamilton sixth. Kimi Antonelli had one of the best drives of the day, climbing from sixteenth to fifth.

The result extends Russell's championship lead and denies Verstappen his first win of the season."""

def edit(t, every=4):
    w = t.split()
    return " ".join(random.choice(["the","a","very","also"]) if i % every == 0 else x
                    for i, x in enumerate(w))

rows = {"WM long": [], "Plain long": [], "WM short (20 words)": [], "WM 25% edited": []}
for p in prompts:
    wm, plain = gen(p, True), gen(p, False)
    rows["WM long"].append(score(wm))
    rows["Plain long"].append(score(plain))
    rows["WM short (20 words)"].append(score(" ".join(wm.split()[:20])))
    rows["WM 25% edited"].append(score(edit(wm)))

for k, v in rows.items():
    print(f"{k:22} mean={sum(v)/len(v):.3f}  runs={[round(x,3) for x in v]}")
print(f"{'Human text (F1)':22} {score(human):.3f}")

Results

These scores are from one run. Generation and word replacement are random, so reruns can produce different values.

Text	Average score	Range across five prompts
Watermarked responses	0.564	0.548 to 0.586
Plain GPT-2 responses	0.497	0.489 to 0.509
First 20 words of watermarked responses	0.573	0.545 to 0.607
Watermarked responses with every fourth word edited	0.500	0.495 to 0.504
F1 text sample	0.498	One sample

The watermarked and plain responses had no overlapping scores in these five runs. The F1 sample scored close to the plain responses. This shows a clear difference in this experiment, though more samples would be needed to establish a reliable detection threshold and measure false positives.

The short excerpts also scored high, but I did not test short unwatermarked excerpts for comparison. These results cannot establish reliable detection on short passages.

Replacing every fourth word brought the average down to 0.500, close to the unwatermarked baseline. This editing pattern weakened the signal measured by this detector. It does not establish that changing any 25% of a passage would have the same effect.

This experiment used GPT-2 and the configuration shown above. It does not measure the performance of Claude's watermark. A detected watermark provides evidence that the matching watermarking system was involved in producing the text. Failing to detect one does not establish that a person wrote it.

Sources

Our SQL Server loop kept writing the previous row's value

Amit — Sun, 28 Jun 2026 16:08:34 +0000

A loop that looks correct, passes review, and still writes the wrong value, because of two documented T-SQL behaviors.

Here's a loop that resolves a value for each item: read it from a primary table, fall back to a secondary one when the primary has nothing, write the result. It looks correct. On the wrong data it writes the wrong value, and the wrong value isn't random: each bad row gets the value belonging to the row processed just before it.

No error, no exception. The loop reads correctly top to bottom. The cause is two T-SQL behaviors that Microsoft documents and that are easy to misread:

A variable declared inside a loop is set to NULL once for the whole batch, not once per iteration.
SELECT @var = column leaves the variable alone when the query returns no rows.

Either one is survivable alone. Together they make a missing row reuse the last good answer.

What the code looks like

WHILE (@cnt > 0)
BEGIN
    DECLARE @Value VARCHAR(100)
    SELECT @Item_Code = ITEM FROM @ItemList WHERE RowNo = @cnt

    SELECT @Value = isnull(p.ITEM_VAL, f.ITEM_VAL)
    FROM @Primary p
    LEFT JOIN @Fallback AS f ON p.ITEM_CODE = f.ITEM_CODE
    WHERE p.ITEM_CODE = @Item_Code
      AND p.ITEM_SEQ = (SELECT MAX(pmax.ITEM_SEQ) FROM @Primary pmax WHERE pmax.ITEM_CODE = @Item_Code)

    IF (@Value IS NULL) OR LTRIM(RTRIM(@Value)) = ''
    BEGIN
        SELECT TOP 1 @Value = ITEM_VAL FROM @Fallback WHERE ITEM_CODE = @Item_Code
    END

    UPDATE @Target SET ITEM_VAL = @Value WHERE ITEM_CODE = @Item_Code
    SET @cnt = @cnt - 1
END

Read top to bottom, the intent is plain: resolve from the primary table, fall back to the secondary one, write the result. The DECLARE @Value inside the loop looks like it hands each pass a clean variable. That assumption is the trap.

A DECLARE inside a loop isn't a reset

In C++, Java, or C#, a variable declared inside a loop body is a fresh binding every pass. T-SQL doesn't do that. A local variable's scope is the whole batch, and it's initialized to NULL when it's declared, not at the point in the code where you wrote DECLARE.

So putting DECLARE @Value inside the WHILE is just decoration. There's one @Value for the whole run, set to NULL once, and every iteration after the first inherits whatever the previous one left in it. That's also why SQL Server doesn't complain about a double declaration even though the DECLARE line is reached on every pass; it isn't a statement that re-runs each time.

A quick proof:

DECLARE @i INT = 0
WHILE @i < 3
BEGIN
    DECLARE @x INT
    SET @x = ISNULL(@x, 0) + 1
    PRINT @x
    SET @i = @i + 1
END

It prints 1, 2, 3. If the DECLARE reset @x each pass, you'd get 1, 1, 1.

Docs: DECLARE @local_variable (Transact-SQL).

SELECT-assignment does nothing when nothing matches

The resolve uses SELECT @Value = .... When the item isn't in the primary table, the WHERE returns no rows. You'd assume that sets @Value to NULL. It doesn't. With SELECT-style assignment, a zero-row result leaves the variable exactly where it was.

So on a missing item the assignment runs, changes nothing, and @Value still holds whatever the previous item resolved to.

Microsoft documents this exact case in Example A on that page: a variable set to a default, a SELECT @var = ... WHERE that matches nothing, and the variable keeps the default.

Docs: SELECT @local_variable (Transact-SQL).

How the two combine

Now the failure is just mechanics:

@Value is never cleared between iterations.
For an item missing from the primary table, the resolve matches nothing and doesn't touch @Value.
The guard IF (@Value IS NULL) OR LTRIM(RTRIM(@Value)) = '' sees a leftover non-null value, decides resolution already worked, and skips the fallback.
The item gets written with the previous item's value.

The fallback that should have rescued the missing item never fires, because the stale value makes it look like there was nothing to rescue.

Fixing it

Two changes close the gap.

Clear the variable at the top of every iteration so a missing row can't carry the last value forward:

WHILE (@cnt > 0)
BEGIN
    SET @Value = NULL
    ...
END

And for a single-value lookup like this, prefer SET with a scalar subquery over SELECT-assignment. A scalar subquery that finds no rows gives you NULL, which is what you want here, because then the IS NULL guard does its job:

SET @Value = (
    SELECT isnull(p.ITEM_VAL, f.ITEM_VAL)
    FROM @Primary p
    LEFT JOIN @Fallback f ON p.ITEM_CODE = f.ITEM_CODE
    WHERE p.ITEM_CODE = @Item_Code
      AND p.ITEM_SEQ = (SELECT MAX(pmax.ITEM_SEQ) FROM @Primary pmax WHERE pmax.ITEM_CODE = @Item_Code)
)

It reads fine. Each behavior is harmless on its own, which is why the pair survives review. If a loop variable can outlive its iteration, assume it does.