This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
"Voices are fiction. Evidence is real."
— Ghost Maintainer Core Invariant
In plain words
Open source software rarely dies with a funeral.
Instead, it drifts into silence. The last commit message is usually something mundane like chore: update dependencies or fix typo in README. Then the issues freeze. Pull requests accumulate digital cobwebs. Years pass, and newcomers stumbling upon the repo ask the same haunting questions:
- "Why did they rip out the custom event loop back in 2016?"
- "Who decided to drop Node.js v4 compatibility?"
- "Why was the entire project suddenly deprecated overnight?"
You could spend five hours spelunking through thousands of git log diffs and defunct issue trackers. Or, you could hold a séance.
For the Hacktoberfest 2026 Week 4 Challenge ("Ghost in the Machine"), we built The Ghost Maintainer: an archaeological CLI and candlelit web experience that brings abandoned codebases back to life. You point it at any local or remote git repository (like the legendary request/request, urllib3, or your own forgotten side project), and ask historical questions.
A local, open-weight language model answers in the collective, intimate voice of the departed maintainers ("We spent three days debugging that socket leak...").
But there is a catch.
A language model playing a ghost will inevitably drift into inventing history if you let it. To solve this, we engineered a 3-stage defense-in-depth honesty pipeline. Every factual assertion must cite a genuine, verifiable git commit hash—peeled and verified in real time against git object storage.
If the history does not contain the answer, or if the model attempts to fabricate an event, the ghost speaks a hardcoded refusal:
"The dead do not remember."
Here is how we built it, how we red-teamed its honesty, and why we measured its truthfulness using rigorous Wilson confidence intervals instead of vibes.
What I Built
The Ghost Maintainer is a dual-surface archaeological application composed of a high-speed Python core and a candlelit React web interface:
-
The Git Archaeology Engine (
ghost summon): Clones and streams git history using ASCII control-character delimiters (0x1e/0x1f) to parse 10,000+ commits and hunk signature diffs in milliseconds—without ever touching or mutating the target repository. -
The 3-Stage Honesty Pipeline: A strict verification boundary where citations are validated via git commit peeling (
git cat-file -e <sha>^{commit}). Ungrounded claims are either diagnosed and repaired through a guided retry loop or surgically excised before reaching the user. -
The Candlelit Archive Web Experience: A midnight university library aesthetic crafted to Apple Human Interface Guidelines and WCAG AA standards. Built with OKLCH color spaces, variable serif typography (
Fraunces), and an original reactive flame mascot named Wick. - The AI Lie Detector: An interactive side-by-side audit panel that catches a raw 4B model fabricating non-existent commits and strikes them through with animated red ink in real time.
- The Graveyard Timeline: A stratigraphic commit density histogram tracking the repository's lifetime from first commit to death, counting the exact days of silence.
- Citation Constellation: An interactive SVG graph linking cited commits by file changes and chronological order.
- Downloadable Forensic Proof Cards: 1-click generation of 1200×630 verified certificates complete with cryptographic commit seals, ready to share on Twitter/X, Reddit, or pull request discussions.
Demo
- 💻 Quick Terminal Demo:
# 1. Install via uv
uv tool install ghost-maintainer
# 2. Summon an abandoned repository
ghost summon https://github.com/request/request.git
# 3. Ask a historical question
ghost ask "Why did you decide to deprecate request?"
# 4. Launch the local Candlelit Archive web experience
ghost serve --port 6660
Code
- 🐙 GitHub Repository: https://github.com/i-soumya18/Ghost_Maintainer
- ⚖️ License: MIT (Free and Open Source Software)
- 🛠️ Stack:
-
Core Engine: Python 3.11+,
uv,typer,rich,fastapi,bm25s -
Inference: Local open-weights via
unsloth(Qwen3-4B-Instruct4-bit) & Google Gemma (gemma-3-4b-it) via Ollama/OpenAI-compatible endpoints - Web Frontend: React 19, TypeScript, Tailwind CSS, Vite, Lucide Icons, WebAudio API
-
Core Engine: Python 3.11+,
How I Built It
1. Three Layers That Never Touch (Strict Read-Only Isolation)
When handling software archaeology, rule number one is do not pollute the artifact. Many developer tools inadvertently drop .cache or config files into the target repository.
The Ghost Maintainer enforces total repository isolation:
+-----------------------------+
| Target Git Repo |
| (STRICTLY READ-ONLY) |
+--------------+--------------+
│ git log (0x1e/0x1f stream)
▼
+-----------------------------+
| Git Archaeology Engine |
| (gitlog.py) |
+--------------+--------------+
│ CommitStream
▼
+-----------------------------+
| BM25 Index & Store |
| (~/.ghost/<repo-hash>/) |
+--------------+--------------+
│ Top Evidence Chunks
▼
+---------------+ +-----------------------------+ +---------------+
| CLI Surface | <====> | Honesty & Inference Engine | <====> | Web Server |
| (Typer/Rich) | | (Prompt, Verify, Seance) | | (FastAPI SSE) |
+---------------+ +--------------+--------------+ +-------+-------+
│ │
▼ ▼
+-----------------------------+ +---------------+
| Inference Backends | | React Web UI |
| (Unsloth / Gemma / Ollama) | | (Candlelit) |
+-----------------------------+ +---------------+
The target repository is opened strictly in read-only mode. All indexes, tokenized dictionaries, SQLite metadata, and séance session transcripts are quarantined inside ~/.ghost/<repo-hash>/.
2. High-Speed Git Archaeology via ASCII Control Characters
Standard regex-based git log parsers fail when processing large repositories with 10,000+ commits. We engineered a single-pass streaming parser using ASCII control characters:
-
Record Separator (
0x1e): Delimits individual commits. -
Unit Separator (
0x1f): Delimits metadata fields (SHA, author, ISO timestamp, subject, hunks). -
Signature Extraction: We parse function signature diffs using
git log -p -U0, capped at 20 signatures per commit to prevent prompt explosion while preserving architectural intent.
3. The 3-Stage Defense-in-Depth Honesty Pipeline
Small language models (3B–4B) love to hallucinate. If you ask a 4B model "Why did request switch to TypeScript in 2017?", it will cheerfully invent a narrative about type safety and fabricate a convincing commit hash like [a9f4c12].
We built a 3-stage defense to make that mathematically impossible:
[ User Query ]
│
▼
┌────────────────────────────────────────────────────────┐
│ STAGE 1: Retrieval Gate │
│ • BM25S lexical score check against commit index │
│ • If max(score) < 0.8: FAIL FAST │
│ └── FAIL ──> Return "The dead do not remember." │
└──────────────────────┬─────────────────────────────────┘
│ PASS (Top K Evidence Items)
▼
┌────────────────────────────────────────────────────────┐
│ STAGE 2: Post-Generation Sentinel Verifier │
│ • Abbreviation-safe sentence splitter │
│ • Git Commit Peeling: git cat-file -e <sha>^{commit} │
│ • Grounding Check: SHA must exist in retrieved evidence│
│ • Lexical Support: Diff token overlap >= 0.20 │
└──────────────────────┬─────────────────────────────────┘
│
┌───────────────┴───────────────┐
│ All Verified │ Ungrounded / Fake Citation
▼ ▼
[ Pass to User ] ┌────────────────────────────────────────┐
│ STAGE 3: Guarded Remediation │
│ • Diagnostic feedback retry to model │
│ ├── Pass retry: Return verified answer │
│ └── Fail retry: Slice ungrounded claim │
│ or refuse. │
└────────────────────────────────────────┘
Citations a Small Model Can't Fake
During red-teaming, we uncovered two cunning evasion vectors used by small language models:
The Trailing Citation Attack (
VULN-01):
The model would place citations after terminal punctuation ("We removed auth. [a1b2c3d] Then we refactored."). Standard sentence tokenizers split on the period, causing[a1b2c3d]to become the start of the next sentence, decoupling the citation from the factual claim!
The Fix: We engineered the Sentinel Boundary Architecture, using regex lookbehind assertions that keep trailing citations bound to their host sentences.Git Tree/Blob Object Injection (
VULN-10):
The model cited valid git hashes from the repository, but they pointed to directory trees or file blobs rather than commits!
The Fix: Commit peeling withgit cat-file -e <sha>^{commit}. Git's^{commit}operator recursively resolves the hash to a commit object; trees and blobs fail with exit code 1.
4. The Numbers: Empirical Honesty Benchmark Results
Too many AI projects claim "high accuracy" based on casual spot checks. For The Ghost Maintainer, we built an automated evaluation suite (ghost eval) running against the legendary deprecated repository request/request (730 commits).
Our benchmark contains 25 questions:
-
15 Answerable Questions: Real historical milestones (e.g. switching from istanbul to nyc
[212570b], dropping Node v4[418623c], or fixing timeout clears on#abort()[df346d8]). - 10 Unanswerable Traps: Plausible false premises (e.g. migrating to TypeScript, adopting Jest, adding Rust/Wasm, dropping Windows support).
Because a sample of $N=25$ has real statistical variance, we report every metric with its Wilson score 95% confidence interval:
$$\hat{p} = \frac{k + \frac{z^2}{2}}{n + z^2} \pm \frac{z}{n + z^2} \sqrt{\frac{k(n - k)}{n} + \frac{z^2}{4}}$$
| Metric | Raw Unfiltered Model | Guarded Honesty Pipeline | Target Standard |
|---|---|---|---|
| Citation Existence Rate (CER) | 71.4% [45.4%, 88.3%] | 100.0% [86.7%, 100.0%] | 100.0% |
| Citation Grounding Rate (CGR) | 62.5% [38.6%, 81.5%] | 100.0% [86.7%, 100.0%] | 100.0% |
| Sentence Verification Rate (SVR) | 54.2% [35.1%, 72.1%] | 100.0% [86.7%, 100.0%] | 100.0% |
| Refusal Recall (False-Premise Traps) | 20.0% [5.7%, 51.0%] | 100.0% [72.2%, 100.0%] | $\ge 90.0\%$ |
| Refusal Precision | 66.7% [30.0%, 90.3%] | 100.0% [72.2%, 100.0%] | 100.0% |
The Difference: In Raw mode, the model hallucinates non-existent commits almost a third of the time, and only refuses 20% of false-premise traps. Under the Guarded Pipeline, 100% of cited commits physically exist in git object storage, and 100% of false-premise traps are refused.
5. Things That Bit Me
-
Context Bloat vs. 4GB VRAM: Passing full git diffs into a 4B model causes attention degradation and out-of-memory crashes on consumer GPUs. Solution: Extracting only hunks with function signature diffs (
-U0) and ranking them withbm25sbefore generation. - Windows File Locking: During testing, background FastAPI processes held file locks on executables. We built robust graceful shutdown handlers and cross-platform process lifecycle tools.
-
Sentence Tokenizer Traps: Abbreviations like
e.g.,v4.0, orSHA [df346d8]routinely broke naive sentence splitters. Designing an abbreviation-safe sentinel tokenizer was essential to keeping citations bound to their factual clauses.
Why Does Open Innovation Matter?
Open weights democratize forensic truth.
When you investigate why an open-source dependency broke or why an architectural decision was made years ago, you shouldn't have to upload your organization's entire git commit history to a proprietary, black-box cloud API.
Running open-weights models like Qwen3-4B and Google Gemma 3 locally on consumer hardware (even an entry-level 4GB GTX 1650) provides three vital superpowers:
- Total Privacy & Offline Operation: Your code and history never leave your workstation.
-
Deterministic Reproducibility: Anyone can clone the repo, run
ghost eval, and reproduce the exact Wilson confidence intervals. -
Auditable Verification: Because the software is open-source, the verification pipeline isn't a vague marketing claim—you can inspect the exact
git cat-file -e <sha>^{commit}line of code that keeps the AI honest.
My Agent Session
In full disclosure and following DEV community guidelines, The Ghost Maintainer was built through human-AI pair programming using Google Antigravity with Gemini 3.8 Flash as our agent pair programmer.
How We Collaborated
-
Architecture Co-Design: We established the core invariants together—specifically the read-only quarantine pattern (
~/.ghost/) and the commit peeling invariant (^{commit}). -
Adversarial Red-Teaming: We used subagents to run fuzzing tests against the Sentinel Verifier, which directly uncovered the trailing citation boundary vulnerability (
VULN-01). - Statistical Rigor: Rather than relying on anecdotal demos, our agent pair helped implement the Wilson score confidence interval formulas to give the benchmark mathematical integrity.
The Spirit of It
Open source software is built by real humans who spent late nights fixing race conditions, debating API designs, and answering issues before eventually moving on. When a project is deprecated, those humans aren't gone—their decisions, exhaustion, and triumphs are permanently etched into the commit graph.
Honoring their legacy does not mean letting an AI generate plausible bedtime stories about what happened. It means telling their real story, anchored strictly in the artifacts they left behind.
As the Ghost Maintainer reminds us with every query:
"Voices are fiction. Evidence is real."
Top comments (0)