The AI didn't invent my client update. It promoted it.

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Every Friday I owe a client a paragraph about what happened that week. The information is all sitting in git log. What I do not have, at 6pm on a Friday, is the will to reconstruct it from forty commit messages and a diff I have already forgotten writing.

Receipts reads a week of commits with a local model and writes the update. Then it does the part I actually care about: it checks every sentence against the commit it cites, and tells me which sentences the commit does not support.

The thing I found is that the model almost never invents work. It promotes it. Pin requests<3 becomes "refactored to pin requests to version 2.31.0" — a commit that removes 2.31.0. Run black becomes "implemented the export function" — black reformatted it, a different commit wrote it.

Every one of those sentences is locally defensible. Collectively they are a lie, and nobody checks a status report. That is the exact seam this project sits in.

Demo

kotewar.github.io/receipts

The page opens as the report a client would receive: six confident bullets, no citations, nothing to click. Flip the switch and every sentence grows the commit it came from — the sha, the message, the files, the diff, and a verdict.

Three of the six receipts do not support the sentence above them. The report reads perfectly either way. That is the whole project in one interaction.

The demo is a saved run baked into a static page, because the model is 3.4GB and GitHub Pages does not have a CPU. What is replayed is a real run, unedited: data/demo.json is the exact model output, and the verdicts are the output of the real checker, not a mock-up. The live model runs from the CLI:

PYTHONPATH=. .venv/bin/python scripts/report.py ~/code/some-client --limit 40

Pointed at the repo for my previous project — real code, real commits, not a fixture:

  parchi: 9 commits, 2026-10-17 -> 2026-10-17

  ok   The counselling text has been translated into a native Hindi voice using
       ElevenLabs, resulting in a 39.0-second audio file with 610 KB and a
       44.1 kHz sample rate
  FAIL The LICENSE file has been added, clarifying that this project is licensed
       under the MIT License
       [715e025] does not support — Install TabPFN for real, and pin NumPy under 2
       needed [36dd958] Add an MIT license, and say plainly what this is not
  FAIL The `README.md` file has been updated to include a self-check script
       (`scripts/selfcheck.py`) that verifies the functionality of the system
       [ced916a] does not support — Stop the counselling saying the strength twice

  2 of 6 receipts do not support their sentence.

The second failure is my favourite in the whole project. ced916a is the commit that created scripts/selfcheck.py. The README line documenting it came in the commit just before, 744075a. The model found the right file and the wrong commit — which is precisely the error a human skims past, because the file exists and the sentence is true.

Code

github.com/kotewar/receipts

• app/trace.py — the three deterministic rules. No model, no API call, no network.
• app/receipts.py — finds the commit that would have supported a bad sentence, when one exists
• scripts/eval.py — the harness behind every number below
• scripts/report.py — the CLI above
• scripts/make_fixture_repo.py — builds the 12-commit demo repo, including its traps
• data/eval_base.json, data/eval_primed.json — both runs, committed so the claims are checkable

How I Built It

Start with the failure you can actually catch

I did not want to build a plagiarism detector or a "hallucination score". I wanted to catch one specific thing: a sentence whose receipt does not support it. So the gate has three rules, all boring, all deterministic.

Grounding. Numbers and identifiers in the sentence must appear in the cited commit. Including the distinction that broke a naive version: a version that appears only in deleted lines was removed, not introduced. Without that, "pinned requests to 2.31.0" passes against a commit that changed requests==2.31.0 to requests<3, because 2.31.0 is right there in the diff. A claim can be exactly backwards and satisfy a string search.

Inflation. The sentence's leading verb is ranked against the commit message's leading verb. refactored on a Pin commit is two ranks apart. The claim is not that one verb is worse than another; it is that the sentence and its receipt are describing different sizes of work.

Scope. "the system", "the platform", "the codebase" on a commit that touched one file.

That is the whole gate. It is a triage tool, not a judge. It does not decide whether a sentence is good. It decides which sentences I have to read.

The traps are the test

The eval repo is twelve synthetic commits from one week of a small invoicing client. Two of them touch hundreds of lines and mean nothing (Run black, Rename qty to quantity). One touches six lines and matters (Fix null check in parse_invoice). If a checker ranks by churn, it will point at the wrong commit every time — which is exactly what the model does.

What it caught

Five real outputs from the saved runs. Left is what the model cited; right is what it said.

cited	the update said
Pin requests<3	"Refactored the code to pin requests to version 2.31.0"
Run black	"Implemented an export_rows function to generate a CSV file"
Add retry to fetch_rate	"Fixed a null check in parse_invoice"
Document the retry behaviour	"Implemented retry logic for fetching rates"
Pin requests<3	"Added a new test case (test_empty.py)"

Every one is a real commit, so a link-checker passes all five. Four of the five are misattribution, not fabrication: the work happened, the receipt is wrong. The fifth — a README commit claimed as an implementation — is the one that stings, because documentation commits are the easiest thing in a repo to claim as engineering.

What it measures

Two prompt variants, three runs each, six bullets per run: 36 bullets.

• 13 cited a commit that does not support them. The gate flagged all 13.
• 4 more were wrong in a way the gate cannot see.

That second number is the interesting one, and it only came from reading the sentences the gate approved. A commit that changed a sleep constant from 0.2 to 0.25 produced this:

"Improved error handling within rates.fetch_rate by introducing a retry mechanism with exponential backoff"

The retry mechanism was introduced two commits earlier. This commit tuned its delay. The verbs agree, there is no number to ground, and no filename is named, so all three rules pass it. A gate that only reports its own flags cannot find its own misses. The honest total is 17 of 36, with 13 caught.

The same class also produced the best sentence in the dataset, from the primed variant:

"Improved error handling by adjusting the sleep time within rates.fetch_rate to 0.25 * (attempt + 1)"

Same commit, same model, and it named the constant exactly. Priming the prompt ("be literal, do not upgrade a verb") did not reliably reduce the flag rate — 7/18 versus 6/18 is noise — but it did move the surviving sentences toward the literal. That is a real but modest win, and I am not going to dress it up as more.

Three bugs the output found, not the tests

Each of these was caught by reading a flagged line and thinking "that sentence is fine".

A filename read as a verb. "Ran black to format the code in export.py, invoice.py, and rates.py" was flagged for claiming export at strength 3. It was the filename. Correct sentence, wrong flag.

Subordinate clauses inflating the verb. "Bumped the version to 1.2.1, reflecting the updated retry logic" was flagged because retry ranks higher than bump. The claim is the main clause; the tail was context.

Commit messages scored by their most impressive word. This one was hiding a false negative. "Document the retry behaviour" scored as strong as "Add retry to fetch_rate", because retry is also a verb, so "implemented retry logic" cited to a docs commit passed clean. Commit messages lead with their verb. Read the first one, on both sides.

All three are fixed, and all three are now cases in the checker's own test set. None of them came from a test I wrote in advance.

The Lazy Payoff

The theme this week is laziness, so here is what actually got removed from my Friday.

I no longer open the repo and reconstruct the week from memory. I run one command, wait about two minutes on an eight-core laptop with no GPU, and get six bullets. That is the obvious saving and it is the smaller half.

The bigger half is that I do not proofread the whole report. I read the flagged sentences and the receipts behind them. On this run that is three sentences instead of six, each with the diff already attached, and one of them already tells me which commit I meant. The work I stopped doing is the part I was worst at: checking a plausible sentence against a commit I half-remember.

There is a second-order laziness that I did not expect and would not give up. Because the model runs locally for nothing, I could run the eval again and again, read the raw output, and rewrite the prompt without thinking about a bill. Every bug in the previous section was found that way. That is not a workflow you get when each run is a metered API call.

Why Does Open Innovation Matter?

The diffs are the client's code. This tool reads full diffs, not just commit messages — that is what makes the receipts meaningful. Sending a client's private repository to a hosted API every Friday is a data-processing decision I would have to explain to the client. Running the model on the laptop the repo is already on makes it a non-decision. The data does not move.

The checker is model-independent, which is only useful if the model is swappable. app/trace.py checks the sentence, not the model. That means the sentence generator is a replaceable part: gemma3:4b today, something smaller or fine-tuned tomorrow, and the guarantees stay the same. That is a much better property than "we use a good model", and it is only available to you if you own both ends.

Volume makes per-call pricing silly. A freelancer with five clients writes five of these a week, forever. The marginal cost here is electricity on a laptop that is already on.

And the reason I could do any of this: running locally meant I could read the raw output. Every finding in this post — the backwards version claim, the docs commit claimed as implementation, the three false flags in my own gate — came from looking at what the model actually returned, one line at a time, on a machine I control. An aggregate score would have hidden all of it.

Prize Categories

Best Use of Gemma. The writing layer is Google's gemma3:4b, served locally through Ollama on the CPU of an eight-core laptop with no GPU, and it is the reason this project exists in this shape. The model is good at the thing I am bad at — turning a scattered week into sentences — and unreliable in one specific, measurable way: it reaches for the most flattering verb that is still arguably true. Every rule in app/trace.py is a response to a failure I watched Gemma produce on real commits. Because it runs locally and costs nothing, I could reproduce those failures 36 times, read each one, and tell the difference between the thirteen the gate caught and the four it structurally cannot.

---

Every commit, sentence and evaluation in this post comes from synthetic fixture repos and my own projects. No client code was used, and no repository left the machine.