This is a submission for the Hacktoberfest Open-Source AI Challenge: Ghost in the Machine.
What I Built
Ghosts of bugs past digs through a project's git history for bug fixes that didn't stick: a bug gets "fixed", then has to be fixed again later.
Some bugs don't stay dead. A commit says "fixed", everyone moves on, and months later another commit fixes the same thing again. Here's a real one from requests, the Python HTTP library:
2012-04-12 · Fix infinite loop on wrong Digest Authentication
2012-07-14 · Fix infinite loop on wrong Digest Authentication (Issue #547)
"Modified code to use the current fix versus the old fix, which was broken."
Same title, three months apart. The first fix didn't hold.
How it works:
-
Git finds the suspects. For every commit that says it fixes something,
git blamechecks whose lines it rewrote. If those lines came from an earlier fix, that fix may not have held. No AI here, just git. - Gemma judges. A local Gemma 3 model, running on my own PC through Ollama, reads both commit messages and both diffs and picks one: "same bug came back", "the fix caused a new bug", or "different problem".
- Plain checks keep it honest. (This is the part I didn't expect to need. See below.)
Demo
I ran it on click and requests, the top 12 suspects in each, with Gemma 3 12B. Each run took about 4 to 5 minutes on my PC. It writes a graveyard page:
Here's the clearest ghost it found, in click:
BURIED 2026-04-29 19fd4d6 "Fix: Ensure fish completion handles multiline help strings correctly (#3126)"
RETURNED 2026-05-23 c353590 "Fix broken fish completion and multiline help string"
23 days in the grave. The second fix rewrote 58 lines the first one wrote.
Git receipt: both messages mention #3126
"The echo fades, the shell remains."
The later commit says it plainly: "Fix regression introduced in 8.4.0 by #3126 and properly fix #3043". Fixed, broken, fixed again.
Gemma called 14 ghosts across the two projects. I checked every one by hand:
- 1 is clearly real: the fish completion one above.
- 6 might be. Same function, similar symptom, years apart, like click's short-help truncation getting fixed in 2014, 2021 and 2026. But the commit messages don't say the old fix failed, so I can't prove it.
- 7 are not. They're different bugs in the same code, or an earlier fix that broke something else.
Every verdict is in HAND_CHECK.md.
Only 2 of the 14 had a git receipt: the real ghost, and a fix that caused a different bug. So here's what I'd trust: git is great at finding suspects, the receipt is the strongest clue, and Gemma is a decent first reader but a bad judge.
Code
Ghosts of bugs past
Some bugs don't stay dead. A commit says "fixed", the bug gets buried, and months later another commit says "fix it again". This tool digs those up.
-
Git finds the suspects. For every commit that says it fixes something,
git blamechecks whose lines it rewrote. If those lines came from an earlier fix, the earlier fix may not have held. - A local open model judges. Gemma (through Ollama, on your own machine) reads both commit messages and both diffs, and says whether it's the same bug coming back.
- A plain checker keeps the model honest. It must quote both commit messages word for word, and it may not invent numbers. If it fails, it gets one retry with the reason, then the answer is dropped.
- It has to say it twice. The verdict is a choice between two phrases, "same bug came back" or…
I built it with Claude Code helping me write the code and read the diffs. One Python file, no dependencies besides git and Ollama:
ollama pull gemma3:12b
git clone https://github.com/pallets/click
python ghosts.py click
How I Built It: Gemma is a yes-man
The first version asked Gemma 3 4B a simple question about each suspect: is this the same bug? I ran it on click.
It said yes to all 15.
Including pairs of commits that only touched the docs. Its reasons sounded great every time: "the later commit essentially reverts the changes made in the earlier commit..." It wasn't true.
So I added a second question: "do these fix different problems?" A real ghost should get "no". Gemma said "yes" to that too: every time it had said "same bug", it now also said "different". Zero ghosts.
A small model leans towards "yes", whatever the question. So I changed three things:
- No more yes/no. Gemma picks one of three phrases. I added "the fix caused a new bug" because without it, Gemma filed those under "same bug".
- Ask twice, in a different order. The second time, the choices come in reverse order. Randomness is off, so an answer Gemma is sure of comes out the same both times. If it changes, the answer is dropped. This doesn't make Gemma right: a confidently wrong answer still passes. It only throws out the ones it's unsure of.
- Check it against answers I know. I read 8 pairs of commits from click and requests myself and labelled them: 2 real ghosts, 2 fixes that caused a new bug, 4 different problems. Then I scored each setup:
| Model | Sees the code changes | Real ghosts found | False ghosts |
|---|---|---|---|
| gemma3:4b | yes | 2 of 2 | 5 of 6 |
| gemma3:4b | no | 1 of 2 | 2 of 6 |
| gemma3:12b | no | 2 of 2 | 4 of 6 |
| gemma3:12b | yes | 2 of 2 | 1 of 6 |
Eight pairs is small, and I tuned the prompts while looking at them, so this is a sanity check, not a benchmark. Even 12B still says "same bug" first most of the time. Asking twice is what catches it.
Two more plain checks from git: Gemma has to quote both commit messages word for word. If it can't, it gets one retry, then its answer is thrown out. And every ghost shows a git receipt: does the later commit point back at the earlier one (its id, a shared issue number, or words like "regression" or "broken")? That part needs no AI at all.
This whole thing started with Rohit's post about TJ Holowaychuk's surviving lines, where Gemma called 3,527 commits "3,527 silent years". Same lesson here: the model writes well. The checks are what you can trust.
Why open source AI
Your git history is your team's private diary. With Ollama and Gemma it never leaves the machine: no API key, no cost per commit, and anyone can rerun my numbers with python eval.py.
The Spirit of It
A bug that gets "fixed" and comes back is a ghost: something the code was supposed to let go of, still walking the halls. This tool doesn't exorcise anything. It just reads the headstones, and it's honest about when it isn't sure.
Top comments (1)
Nice article! Good luck with Hacktoberfest!