Turns out I'm the one making the ghosts💀

This is a submission for the Hacktoberfest Open-Source AI Challenge: Ghost in the Machine

---

On 8 August I committed a file called seed.sql. On 25 October I filled it in.

For 78 days it was zero bytes. Nothing ever told me🙃. The file existed. The import ran. The app worked. Only the data was missing — and an empty file doesn't raise an error. It just sits there being nothing, very quietly, for two and a half months.

I found out while deploying, a few days ago. So I built something to go looking for whatever else was in there.🤔🤓

What I Built

Ghosts in the Machine reads a public repository's git history and finds failures that never produced an error — the ones nobody noticed, because there was nothing to notice.

Point it at a repo. It walks the commits, finds what broke silently, and an open-weight model groups it, ranks it, and writes up what happened.

   a public repo
        │
        ▼
 ┌─────────────────┐
 │  walk history   │   every commit, every diff
 └────────┬────────┘
          ▼
 ┌─────────────────┐
 │  rules          │   no AI. zero-byte blobs,
 │  (deterministic)│   credential-shaped lines
 └────────┬────────┘
          ▼
 ┌─────────────────┐
 │  oddness        │   no AI. size collapses, resurrected
 │  (deterministic)│   paths, diffs that don't match
 └────────┬────────┘   their own message
          ▼  findings, with commit hashes
 ┌─────────────────┐
 │  Gemma          │   did this break? which of these
 │  (open weights) │   matter? how do I explain it?
 └────────┬────────┘
          ▼
 ┌─────────────────────────────────────┐
 │  what the model made of it          │
 ├─────────────────────────────────────┤
 │  what git actually recorded         │  ← both shown, so you can
 └─────────────────────────────────────┘    check one against the other

When I ran it on my own repositories:

• seed.sql — committed empty, filled 78 days later
• requirements.txt — committed empty in the initial commit, filled 79 days later
• readme.md — committed empty, by a commit whose message was "Add README"
• A database password in three places in one commit, one of them commented out
• A manifest.json, empty, which I later deleted without ever noticing it held nothing

Every one of those is mine. None of them ever failed.

How I read the theme

The organisers suggested a haunted terminal and an offline oral history tool. I went elsewhere. I kept coming back to what a ghost actually is: something present, that nobody can see, that everyone has stopped looking for.

That describes a file committed empty exactly. It's in the tree. It's in every clone. It's in the history permanently. And because it never errors, nobody is looking.

The repository is the haunted house. The commits are the record of who lived there. I just read the record.

Who this is for

The case I kept thinking about isn't mine. It's inheriting a repo.

You join a team, or pick up something abandoned, and you want to know: what's wrong with this that nobody is going to tell me? The README is aspirational. The code is current. The history holds the real answers, and nobody reads 500 commits to find them.

Nobody hands over a project with a list of everything quietly broken in it. Partly because it would be awkward. Mostly because they don't know either. That's the whole point — these failures produced no error, so the person who made them is as unaware as the person inheriting them.

The same goes for open-sourcing something, or handing a repo to a client. What's in the history that shouldn't be? Secrets survive deletion, and people usually find that out afterwards.

Demo

→ ghostsinthemachine.tech · also at ghosts-in-the-machine.onrender.com

Two exhibits load on arrival, from my own repositories. Paste in any public repo and it will read that one instead.

It's a free instance, so the first request after a quiet spell takes a moment to wake up. Small and medium repos come back in under a minute. Large ones take longer, because cloning dominates.

Code

→ github.com/VidhiDixit2000/ghosts-in-the-machine

How I Built It

Two layers, deliberately separated.

The deterministic layer walks the history and emits findings. No AI. It checks whether a blob is zero bytes when the file extension implies content, and whether added lines look like credentials. Facts, with commit hashes attached.

The model layer answers a different question: which of these matter, which are the same story, and how do I explain them to a person? That's Gemma, via OpenRouter — open weights, hosted inference, so the demo runs without a GPU.

The split isn't tidiness. Each layer does what the other is bad at.

A regular expression can't tell you an empty file mattered. It has no idea whether 78 days is a long time. Ask a model to scan a whole history instead and it invents findings, misses real ones, and burns tokens doing it.

Here's the split earning its keep. One repo produced three separate findings: a connection string in database.py, a DATABASE_URL in docker-compose.yml, and a POSTGRES_PASSWORD in the same file. Three true findings. One story — the same password, three places, one commit. The rules found them. Only the model saw they were one ghost.

On the page, both are shown. What the model made of it sits above what git actually recorded, so you read the narration and then check it against the hashes underneath.

That mattered more than I expected.

The model was confidently wrong, and I could prove it😏

While I was tuning the narration prompt, Gemma 3 12B wrote this about the credentials:

These values... were removed from the working tree before the commit.
However, they remain present in the repository's history.

That's impossible. Git history contains what was committed. If something was removed before a commit, that commit never recorded it — it wouldn't be in the history at all. The sentence contradicts itself.

This was google/gemma-3-12b-it at temperature 0.3.

What actually happened: committed 30 July, removed 25 October. Nearly three months apart. The findings said so correctly. The model garbled them on the way to prose.

I only noticed because the evidence was sitting right underneath🤷🏻‍♀️. If the narration had replaced the findings instead of sitting beside them, I'd have shipped a well-written falsehood about my own repository.

Moving to Gemma 3 27B and lowering the temperature fixed that particular error. But the architecture caught it, not the model choice.

If you're building on a model: test across versions and temperatures. A wrong answer at 0.3 looks completely reasonable until you check it against something that can't be wrong😂.

The mistake I made, and what I did about it

I started by writing rules for failures I'd personally hit. Empty files, because of seed.sql. Leaked credentials, because of a .dockerignore I forgot and a password I thought I'd deleted by commenting it out.

Then I saw what I'd built: a tool that finds failures someone has already met. For a project about failures nobody noticed, that's close to self-defeating. A rule per failure type, forever, and only ever the ones already on the list.

The way out isn't more rules. There are infinitely many. It's changing what the rules are for.

Git's vocabulary is tiny🤏🏻. A file appeared, vanished, changed, or moved. Its bytes grew or shrank. That's nearly all of it. So however many ways software can break, a failure has to express itself in that vocabulary to be visible at all — and it lands in one of a few shapes:

• something got smaller when it shouldn't have
• something vanished and came back
• two paths that should have been one
• a diff that doesn't match what its message claims
• a change that undid an earlier change

Truncated JSON and a gutted requirements.txt are different failures with the same shape. A case-only rename and a resurrected file are different failures with the same shape.

So I added a second deterministic layer that names nothing. Six measurements: size collapse, a file too small for its extension, a resurrected path, two paths differing only in case, a large diff under a short message, a change that's almost all deletion. Each scores how unusual a commit is. The dozen oddest go to the model in one call, and the model says what — if anything — went wrong.

Like a metal detector. It doesn't know what a coin is. It knows there's metal here. Someone else digs.

The signal set is bounded by what git can record. What the model can name isn't.

Here it is working. On andela/buggy-python,
three commits long, the oddness layer flagged a .json file deleted inside a 14,000-line commit. Neither rule would have caught it — not empty, not a credential. It surfaced because it was strange.

And then the model explained it as an attempt to hide evidence.

Both halves of that sentence are the point.

It also found something in my own repository that I'd never have written a rule for🫢. My README describes GitHub Actions. No commit in the history implements them. The measurements only noticed that a large change arrived under a short commit message — they have no concept of documentation drift. The model read the change and named it.

The narration varies between runs. A later run described the same two commits as a formatting change instead. The findings underneath don't move.

To be honest about it: those six measurements are still rules. The difference is how much each one buys.

My first rule — the one that looks for empty files — finds empty files. That is all it will ever find. But "this file lost most of its contents" isn't tied to any failure. It catches a truncated JSON, a gutted requirements.txt, a half-saved migration, a bad merge. One measurement, many failures, including ones I've never seen.

The rules didn't go away. They stopped growing every time software finds a new way to break quietly.

Why Open Innovation Matters

This tool reads your repository. Every commit, every diff, every line that ever held a credential. If it's going to do that, you should be able to read it back: what it looks for, what it sends to a model, what it redacts before sending. The code is public for the same reason the evidence sits beside the narration. A tool making claims about your history should be checkable.

Open weights meant I could reason about failure. When Gemma produced that impossible sentence, I could see which model, which version, what temperature, what prompt. I changed two of them and watched the failure go away. That's a debugging loop. Behind an opaque API it would have been a shrug.

What It Doesn't Do

• Two named rules, six measurements. Most well-maintained repositories come back empty, correctly. The page says so rather than pretending otherwise.

• It doesn't read a whole repository. Commit depth is capped, and the page tells you when a history was truncated.

• Large repos are slow. Cloning dominates, not scanning.

• The model reads intent into odd commits, until you stop it. On a repo called buggy-python, with a commit message reading "write buggy code", it described a deleted JSON file as an attempt to hide evidence. The measurement was right that something unusual happened. Everything after that was the model writing a story. Telling it not to speculate removed the story. Telling it not to invent sequence did not — until I stopped banning the words I didn't want and prescribed the ones I did.

• The narration isn't stable. Same repo, same prompt, same temperature: one run called a deletion "possibly intentional experimentation", the next called it "a deliberate attempt to hide evidence". No code changed in between.

• It still summarises badly. I could stop it inventing things. I couldn't stop it garbling them. In one run it wrote that a file was "added and deleted in the same commit" — and then, in the very next sentence, said the file had existed in an earlier commit. Both can't be true. The findings it was given said the right thing: added in one commit, deleted in a later one. The model had the facts and mangled them while squeezing them into two short paragraphs.

• Secret detection is shallow next to dedicated tools. The quiet-failure half is where the gap is.

Every one of those is visible on the page, because the evidence sits under the narration. That isn't a coincidence. It's why the page is built that way.

What I'd Build Next

Some named rules will always be worth keeping. A dependency manifest losing most of its contents is worth catching directly rather than inferring. But they shouldn't live in one growing file of conditionals.

The shape that scales is a small registry. Every rule implements the same interface — does this apply to this change, and if so what does it find — and every rule returns findings in one format. Adding a rule then costs nothing structurally. Rules that don't apply to a file never run against it. Each commit is parsed once and shared. Duplicate findings collapse by fingerprint. One rule failing doesn't take the scan down.

A registry to make named rules cheap, and oddness scoring to make most of them unnecessary. The first bounds the maintenance cost. The second bounds how many you need at all.

The Spirit of It

I'm the kind of person who commits functionality in pieces. Somewhere in all that pushing, something ends up in a file that shouldn't be there. There are mistakes in every repo I've shipped that I'd only see if I stepped away, came back, looked properly, and went wait, what?

So I built something to look. It spent most of its time looking at mine.

The one that got me wasn't the empty files. It was the password. I'd found it weeks earlier — a connection string in database.py — and I commented it out, felt responsible, moved on. My own tool later showed me that same line, in a commit from July, with the # still on the front. The comment character meant nothing. Git had kept it the whole time, and so had every clone anyone had ever made.

Then the same password twice more, in a file I hadn't thought about. Three copies of one mistake, and I'd only ever found one.

Somewhere in there it stopped feeling like I was finding ghosts and started feeling like I'd been making them.

The 78-day empty file is the one I tell people about, but it isn't really a mistake. Nobody did anything wrong. A file was committed before it had content, and there was no moment at which anything would have said so. It sat there, in every clone, for two and a half months, being nothing.

That's what I find unsettling about git. Your repository remembers everything you did and tells you none of it unless you go and ask. Every project I've shipped has things in it I don't know about yet.

Which means the next person to open one of my repos wouldn't know either. That part I hadn't thought about until I was looking at the output.

Building a ghost detector and finding five of your own is a strange way to spend a couple of days. I commit differently now.

Prize Categories

• Best Use of Gemma — Gemma 3 27B (open weights), served through OpenRouter, does all the judging, grouping and narration. The project does not work without it.
• Best Use of Render — Render is the runtime: the web app and every inference call run from a single auto-deploying service.
• Best Use of ElevenLabs — ElevenLabs generates the spoken introduction and the ambient room tone on the page.
• Best Use of a .tech Domain — live at ghostsinthemachine.tech
• Best Use of GitHub Copilot — a GitHub Actions workflow runs the tool against its own repository on every push, so the project is continuously scanned by itself. Recent runs.

The rest is Python, FastAPI and GitPython: history walking, the rules, the oddness measurements and the web app.