A deleted file isn't gone. It's waiting to be overwritten

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

Recover deleted files from a disk image and narrate what they were, with open-weight models running entirely on your machine
Repository: github.com/flameeecyber/palimpsest · MIT · Python 3.11+ · Gemma 3 4B + nomic-embed-text via Ollama · no network required.*

What I Built

The letter is 208 bytes, and it is fictional. But the situation isn't.
I built a 128 MB ext4 disk image, wrote the letter into it, and deleted it at 07:34 on a Tuesday morning. Then I tried to recover it.
The filesystem still knew the letter existed. The directory entry survived. The inode survived. Even the deletion timestamp survived. icat can copy the bytes out. Any classical recovery tool — photorec, scalpel, foremost — will hand the letter back, perfectly, and then stop.
Because the next question is not a filesystem question.
What else went with it? Which of these eight photographs was in the box? What was the file whose name survived but whose blocks did not? A recovery report that lists only what it recovered is lying by omission.
That gap is Palimpsest. It recovers deleted files from a disk image and then explains what they were, using open-weight models running entirely on your own machine. It looks at the photographs, groups the files by meaning, writes a labelled hypothesis for the ones that cannot be recovered, and composes an epitaph for each artifact in which every sentence carries the id of the evidence it came from.
It is not a chatbot for your files. It is a witness statement with a chain of custody.

The demo

The sample case, rendered — nine recovered files, two hollow tombstones, and the evidence table behind every sentence. Everything on that page was generated on this laptop from a synthetic disk image, and the four commands below reproduce it:
./scripts/make_sample_image.sh # builds a 128 MB ext4 image, no root, no mount
palimpsest run sample/ghost.img --out out
palimpsest ablate --out out
palimpsest verify --out out
out/site/index.html is a static site: no server, no JavaScript, no CDN, no webfont. Copy it to a USB stick and it still opens.

The rendered case. The two dashed stones at the bottom are files that provably existed and did not come back.

How it works

Recovery first, because the story depends on it. No root, no loop device, no mount — the image is built with mkfs.ext4 and debugfs, so it's reproducible in CI and can't damage the machine it runs on:
fls -r -d -p image # every deleted entry, recursively
istat image # the ext4 "Deleted:" line — when the file died
icat image # the bytes, if the blocks are still free
One detail matters: istat gives the deletion time, not the modification time — the difference between when this was last written and when someone got rid of it. And when icat returns zero bytes, the artifact is kept as missing, with its name, size, parent directory and deletion time. It is not dropped from the report.
From there, five model stages:
1. Triage. Gemma 3 looks at the pixels of a recovered image and returns subject, setting, people count, visible text, and a human label. A carved file has no name and no EXIF, so the pixels are the only evidence. On this CPU, one cold vision call takes 200–700 seconds, so triage is capped at the most promising images — a scheduling decision, not a filter that hides failure.
2. Link. Every artifact is rendered as a line of text, embedded locally with nomic-embed-text, and clustered with union-find over a cosine graph. No random seeds, no hidden parameters. The model names each group the way a person labels a box of belongings — C1: Summer Family Photos, C2: Sunset Landscapes June.
3. Infer. For a file whose blocks are gone, the model gets only facts: name, size, deletion time, directory, neighbours. It returns a one-sentence hypothesis. The UI renders it as a dashed tombstone labelled hypothesis, never as a recovered file. photorec reports an unrecoverable file as a failure. Palimpsest reports the shape of the hole.
4. Narrate. Two to four sentences per artifact, each citing evidence ids. Then the validator runs.
5. Query. palimpsest ask embeds your question, retrieves the ten nearest artifacts, and answers with artifact citations — then checks the answer the same way the epitaphs are checked.

A citation is not support

The first version of the validator checked that every cited evidence id existed. That is the obvious check, and it is not enough.
gemma3:1b wrote:
"The image depicts a sunset over the sea." — citing E1: IMG_2019_0006.png
Every word is plausible. None of it came from the evidence. A citation-existence check published it.
So the second check computes a support score: the fraction of the sentence's content words that actually appear in the cited values, after removing function words and generic visual nouns. Below 0.2, the sentence is struck. Years get a stricter rule: a year in the sentence must appear verbatim in a cited value.
Struck sentences are never published. They appear on the detail page crossed out, with the reason — because the failure is evidence too.

The same letter, narrated by gemma3:1b instead of gemma3:4b. One sentence survives with its source at 100%; the other quoted the letter but cited the wrong row, so it is struck on the page with the reason. A forensic report that silently deletes its failures is not a forensic report.
Then the loop feeds the rejection back to the model, once, with the exact strike reason. The best attempt wins. That retry loop is the difference between a demo and something that holds up when the model is having a bad day.
I want to be precise about what this is: lexical overlap is a tripwire, not entailment. A fluent sentence can reuse the right nouns and still be wrong. The defence is layered — the prompt forbids invented relations, the support score catches the crude cases, verify re-checks later, and the UI labels speculation differently. It is not a proof. It is a documented, reproducible filter.

The same check, applied to a question

The obvious next feature was a chat box. palimpsest ask embeds the question, takes the cosine top-k over the case, and asks for two to four sentences with artifact ids attached.
The first answer it produced was this:
The recovered photographs [2f3dfa31df], [59125a46a8], [78294c9b10] and [cadaa85ef6] were all created on June 20th, 2019 with an iPhone 8.
Every id is real. Every id is a photograph from the same cluster. One of them was created on June 20th; the others were created on the 15th, the 18th and the 23rd. Retrieval did its job, the citations were correct, and the sentence was false anyway — because a citation is not a check.
So the answer now goes through the same machine as an epitaph. check_answer() splits it into sentences, requires each sentence to cite an artifact that exists, and looks for every timestamp, year, date, and quoted passage in the facts of the files that sentence cites.
The rule that catches the sentence above is the quantifier. If a sentence says all, every, both, each, none or neither, the fact has to hold for every artifact it cites; otherwise one is enough. "The file [2f3dfa31df] was created on June 20th, 2019" passes. The same date stretched over four files is struck, and the strike is fed back once, exactly like a rejected epitaph.
Struck sentences never become the answer. They are printed underneath it, crossed out with the reason. The tool shows you what it refused to say.
The unrecoverable files are always in the context, marked unrecovered, because they have no embedding to retrieve and they are the most interesting thing in the case:
$ palimpsest ask "Which files never came back, and what do you think they were?" --out out

The file [0343d28351] was unrecovered and probably a photograph. The file
[6e82e87c54] was also unrecovered and likely taken on a mobile device.

struck (never shown to the user):
x "Both files have an unknown creation date."
-> cites no artifact
The retry is what the check buys you. Same question, same model, second attempt:
$ palimpsest ask "Which recovered photographs were taken on the same summer
trip, and what does the letter have to do with them?" --out out

The file [c3d3c9aebb] IMG_2021_0002.png was created on 2019-06-16T18:24:01.
The file [fb343c5bd6] IMG_2020_0007.png was created on 2019-06-21T18:29:01.
The file [2f3dfa31df] IMG_2019_0006.png was created on 2019-06-20T18:28:01.
The file [cb023cb531] IMG_2021_0008.png was created on 2019-06-22T18:30:01.
The letter [b81c5c8508] letter.txt mentions a box of photographs from a
summer trip to the coast.
Four true dates instead of one true date repeated four times. Fifteen minutes on this CPU for the two attempts — the honest price of a second opinion when the first one is wrong.

The ablation

The claim "the AI is load-bearing" is easy to make and hard to check, so it is a command:
$ palimpsest ablate --out out

without model with model

artifacts with a human label 0 2
groups found by meaning 0 2
unrecoverable files explained 0 2
artifacts with an epitaph 9 9
sentences written 26 18
sentences with a source 26 18
words about the content 0 17
The baseline is not a strawman. stub_narrate() was written before the model path and emits only what a stat call can prove: "letter.txt is a document of 208 bytes." It writes more sentences and says less. That is the whole argument in one table.

Chain of custody

A forensic report is only worth something if someone else can check it later. palimpsest run writes manifest.json with the source image hash, the SHA-256 of every recovered artifact, the model names, the SHA-256 of every prompt file, and the version of every tool that touched the evidence.
palimpsest verify re-reads the case and checks seven things: the artifact set, every file hash, every citation target, every published sentence's support score, prompt drift, code drift, and the source image hash. Exit code 1 on any failure, so it drops straight into CI.
This is the concrete form of the reproducibility argument. A report produced by a hosted model that was silently updated last Tuesday cannot be reproduced. A report produced by a pinned digest can.

What it does not do

• No write-back. It reports; it does not restore to the original filesystem. That is a different, riskier product.
• No live-disk mode. An image or a directory, never a mounted filesystem. Mounting read-write while recovering is how evidence gets destroyed.
• No encrypted volumes. LUKS, BitLocker and FileVault need a passphrase and a different toolchain.
• No face recognition, no identity inference. The prompt forbids it and there is no code for it. A tool that recovers deleted photographs of people must not also name them.
• No TRIM'd SSD recovery. The blocks are physically gone. Claiming otherwise would be a lie. ## Why open-source AI matters here Three reasons, none of them ideological. The input is the most private object a person owns. A disk holds the deleted photographs, the unsent letter, the medical PDF, the browsing history. Uploading it to a vision API means sending the exact things someone deleted on purpose. A local model is not a preference here; it is the only version of this product a person could ethically run on their own machine, let alone on someone else's. There is no remote path in the code at all — not a flag, not an opt-in, not a fallback. A forensic report has to outlive the model that wrote it. The manifest pins the model name, the prompt hashes, the source hashes and the tool versions. Six months from now, palimpsest verify can tell you whether the case still stands. You cannot do that with a closed model's system prompt, and you cannot do it at all if the model behind an API changes without a version number. The behaviour is editable by the person using it. The prompts are markdown files, not code. Change one line, re-run, and the manifest records that the prompt changed. A model small enough to run on a laptop is also a model you can swap, quantise, fine-tune, or replace next year without asking anyone's permission. And the practical one: it costs nothing to run and works with the network off. No tokens, no rate limit, no account. ## Prize categories I am entering three, because these are the three I actually use:
• Best Use of Gemma — gemma3:4b performs vision triage and evidence-bound narration through local Ollama. During development, gemma3:1b was the measured failure case: it cited a filename for a claim about a sunset, and the validator struck it. No Google Cloud, no API key, no data egress.
• Best Use of Temporal — workflow.py defines seven activities (recover, triage, link, infer, narrate, manifest, render) with heartbeats and retry policies. The case store is the checkpoint, so a worker that dies mid-case resumes at the last completed phase instead of starting over. The case the workflow produced passes palimpsest verify with 39 checks.
• Best Use of Sentry Agent Tracing — tracing.py wraps every stage in a span. With SENTRY_DSN set the spans go to Sentry; with PALIMPSEST_TRACE=1 they print locally, which is how I checked the claim on a machine with no account and no network: $ PALIMPSEST_TRACE=1 palimpsest run sample/ghost.img --out out [span] palimpsest.recover target=sample/ghost.img 347ms [span] palimpsest.enrich count=9 634ms [span] palimpsest.triage count=9 231ms [span] palimpsest.link count=9 736ms [span] palimpsest.infer count=2 0ms [span] palimpsest.names count=9 0ms [span] palimpsest.narrate count=9 2ms [span] palimpsest.manifest artifacts=11 275ms [span] palimpsest.render artifacts=11 1032ms A span knows how many artifacts it saw, never what they were. I am not entering Render, DigitalOcean or .tech: the product is local-first, and putting a real case online would contradict the reason it exists. *