This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine.
What I Built
October has always felt like the end of something and the beginning of something new for me. Every year, it leaves me with a few lessons I didn't expect to learn. Some are about the things I build. Others are about the things I need to leave behind.
This October, I ended up learning something strange about both.
It started with a simple message.
“Greya’s records have been deleted.”
We broke up a year ago. I'd spent months clearing out the little digital reminders: tagged Markdown files in my Obsidian vault, daily notes, shared itineraries, and the list of cafés we'd planned to visit. I checked the folder. The files were gone.
Then I asked my local AI assistant about a trip from last summer. I expected a normal answer about the city.
Instead, it mentioned Greya. And the exact café she loved.
The files were gone. The memory wasn't.
That's a pretty good Halloween story, except the ghost wasn't hiding in an old house. It was hiding in my computer.
So I made Revenant, a small ghost hunter for AI apps. It looks for information you've deleted from one place but that an assistant can still pull from another. More importantly, it checks whether that information comes back later.
The question behind it is simple: “You said you forgot this. Can you still find it?”
I built a deliberately messy test setup. The assistant could read my notes through a separate search memory. Deleting the original files didn't clear every saved snippet. Revenant found the leftovers, showed me where they were, and let me approve their removal.
The next check was clean.
Then I restored an old backup.
Greya came back.
And Revenant caught the same ghost twice.
The Test Run
The best part of the simulation is the moment the green Deleted message stops being trustworthy.
- I add a few fictional travel notes about Greya. The chatbot can answer questions about them.
- I press Delete all. The original files disappear, but an old search-memory copy survives.
- I ask the chatbot about the trip. It still mentions a detail from the deleted notes.
- Revenant finds the surviving snippet, points to its source, and asks me to approve removal.
- I run the check again. The connected memory store no longer returns that snippet.
- I trigger a simulated backup restore. The old snippet reappears, and Revenant raises another warning.
The warning says: “Something you deleted has returned.”
No haunted-house soundtrack needed.
For a repeatable test, I prepared 120 fictional personal notes. Some were exact copies, some were rewritten, and others were harmless notes with similar names or places.
| Test | Result |
|---|---|
| Deleted records correctly identified | 46 of 50 |
| Unrelated records incorrectly flagged | 3 of 70 |
| Known records caught after simulated restore | 10 of 10 |
| Time to flag a restored record | About 4 seconds |
Four deleted notes were missed in this scenario. Three unrelated notes were flagged. I'd rather show those mistakes than pretend the detector knows everything.
How I Built It
I started by giving the same information two homes.
The first was a folder of Markdown notes, similar to an Obsidian vault. The second was a searchable memory store used by the chatbot. That store kept smaller pieces of the notes and their embeddings: numerical representations that help the assistant find relevant text even when the question uses different words.
Here's the problem: removing a Markdown file doesn't necessarily remove a copy that was saved earlier in that second store.
I used a simple setup with a local file folder, a vector search database, and an isolated web dashboard. For every imported note, I kept a source ID alongside its saved chunks. That ID mattered later: Revenant could check whether a snippet still belonged to a file that was supposed to be gone.
The scan worked in two passes.
First, the boring checks. Revenant compared source IDs, deletion records, and exact text fingerprints. If a deleted note still had a saved chunk with the same source ID, I didn't need AI to tell me something was wrong.
Then, the messy checks. A copied memory might have been shortened or rewritten, so an exact text match wouldn't catch it. This is where Gemma, running locally, helped. I gave it a candidate snippet and the relevant deletion context, then asked whether they seemed to describe the same deleted information.
That answer was only a clue, not a verdict. The app still checked the available links between records and marked uncertain cases for review. Gemma couldn't press the delete button.
One early version was too eager. A note about the same city looked like a match even though it was about somebody else. I changed the flow so matching a place name wasn't enough: the warning needed supporting details such as a source link, a matching event, or more than one relevant clue.
That meant fewer dramatic red alerts. Good. A ghost hunter that accuses every shadow isn't very useful.
How the ghost comes back
I made the restore step intentionally predictable. A background script takes an older snapshot of the memory store and puts a previously removed chunk back. Revenant scans again and compares what it finds with its deletion log.
If a deleted record becomes searchable again, the dashboard shows what returned, where it appeared, and which previous deletion it relates to. The removal step still needs approval.
For repeatable checks, a background workflow coordinates the scan-and-verify process and retries failed activities if the memory service is briefly unavailable. A retry doesn't prove that deletion succeeded; the workflow still needs a fresh verification result. The dashboard is hosted separately, while the private matching step remains entirely on the local machine.
I kept the rule that matters most: the AI suggests, but the evidence and the user decide.
There are limits. Revenant only checks systems connected to it. A clean scan doesn't prove that every copy everywhere has vanished. And clearing a searchable memory isn't the same as removing knowledge from a model's trained weights.
Why Does Open Innovation Matter?
There's something odd about uploading private notes to another company's AI service just to ask whether those notes have been forgotten.
I wanted the sensitive part to stay on my computer.
Gemma is an open-weight model, so I can run it locally instead of sending the notes to an outside inference API. I can change the model, test a smaller one, or turn off network access without redesigning the whole investigation.
That doesn't mean AI should do every job. Exact ID checks are faster and more reliable when the original record is still clearly linked. Gemma earns its place when the wording changes and ordinary matching runs out of ideas.
Use simple code for facts you can check. Use a local model for the cases that need interpretation.
The open part isn't just about saving API costs. It means the investigation can happen without sending the very information I'm trying to protect to another service.
My Agent Session
I used an AI coding assistant as a helper for the dashboard structure and some test scaffolding. But I wanted the real proof to come from running the whole story: import, delete, ask, scan, remove, restore, and scan again.
The useful part of the process wasn't getting a pretty dashboard. It was finding out when the detector was wrong, then changing the checks so it had to explain its warnings.
The Spirit of It
I kept thinking about why ghost stories work.
It isn't just that something appears. It's that something you thought was gone shows up again, right when you've stopped looking for it.
In this little simulation, the creepiest moment isn't the first warning. It's the second one. I've already removed the note. I've already seen the green check mark. Then a quiet background restore brings it back.
No magic. Just two parts of a computer disagreeing about what deleted means.
Maybe that's the real ghost in the machine.
If an app says it forgot something, it should be able to show its work.
Prize Categories
- Best Use of Gemma: Local, open-weight AI helps investigate reworded traces without sending private notes to an external model API.
- Best Use of Render: Hosts the isolated dashboard and web app, completely separate from the local private-data scanner.
- Best Use of Temporal: Coordinates repeatable resurrection checks and recovery from temporary background service failures.
Top comments (0)