My submission for the Hacktoberfest Open-Source AI Challenge: Ghost in the Machine. Gemma 3 4B, Ollama and PySide6. Nothing leaves my laptop.
A reminder asks one thing of you: a tap. Snooze, dismiss, "done". None of them checks anything and I am very good at tapping.
So I built Haunt, a small ghost that appears over whatever I am doing, says something unsettling and does not leave until it can see that the work is done.
What it does
You add a task. Every few minutes the ghost appears on top of your other windows and speaks a line written by Gemma. If you ignore it, it comes back angrier: a whisper first, then eerie, then furious.
When you click I did it, the ghost says "Hold still. Let me look at your screen..." and checks. If it does not see the work, it tells you what it does see and it stays. Something like:
I see a YouTube video of cats. That is not "Finish the report".
If you honestly did the work and the model disagrees, an "I swear I did" button appears after two failed claims. A wrong judgement should never trap you.
How it sees
Here is what happens when you click I did it:
- The ghost hides before capturing the screen, so it does not photograph itself.
- Haunt captures visual evidence from the desktop.
- The screenshot and task context are sent to Gemma 3 4B through Ollama.
- The model evaluates whether the screenshot supports the completion claim.
- Haunt processes the verdict. If the evidence is insufficient, the ghost responds and remains part of the interacti on.
The model's job is not to read your mind. It evaluates what is visible on the screen.
That distinction matters: a screenshot is evidence of a moment, not proof of everything you have done.
Not everything needs a model
If a task is tied to a file or a repo, Haunt does not need to look at anything:
python haunt.py add "Write tomorrow's notes" --proof file:~/notes.txt
python haunt.py add "Commit the lab work" --proof git:~/lab
The first finishes the moment the file is saved after the task was created. The second finishes when a new commit lands. If a file was saved, I do not need a 4B vision model to tell me. Checking that is cheaper and more reliable than asking a model to interpret pixels, so the model only handles the tasks that really are visual.
Does the judge work?
I tested the judge using 10 screenshots: 5 representing completed tasks and 5 representing incomplete tasks.
This was an initial sanity check to see whether the model could distinguish relevant work from unrelated desktop activity.
I did not retain the individual verdict counts needed to report a reliable confusion matrix, so I am not presenting invented accuracy numbers. Ten screenshots would also be far too small a dataset to establish general reliability.
A more rigorous evaluation would use a larger, labelled dataset and report true positives, false positives, true negatives, false negatives, precision and recall.
What broke
The buttons did nothing. The ghost's buttons looked fine and did not respond to clicks. The tests were green the whole time, because they click by calling button.click() in code, which skips the window system entirely. I isolated it with a minimal Qt button test and then the real window. The culprit was the Qt.Tool window flag. Removing it fixed clicking in my WSLg setup. I only tested that one setup, so other desktops may behave differently.
A newer PySide6 crashed the app. PySide6 6.12.0 crashed Haunt on my machine. I found that 6.8.3 works in a clean environment and pinned it in requirements.txt.
localhost was not where Ollama lived. I develop in WSL while Ollama runs on the Windows host, and localhost inside WSL did not reach it. I point Haunt at the host's address with OLLAMA_HOST. That address is specific to my setup.
The "voice" was a beep. My first version called QApplication.beep() and I was calling that speech. The real fix was Windows' built-in System.Speech, driven from PowerShell on a background thread so the ghost never freezes while it talks.
Why it had to be local
This project exists because of a conflict. A ghost that watches your screen sees your mail, your chats and whatever you would rather not show anyone. That is a reasonable thing to build only if the screenshot goes to your own machine and nowhere else.
- Privacy. The screenshot is held in memory and sent to Ollama on my own computer. After the one-time model download, nothing in Haunt calls out to the internet and there is no API key.
- Every look is free. A ghost that checks you repeatedly is only affordable when a look costs nothing.
-
I can swap the model.
HAUNT_MODELor--modelpicks any vision-capable model fromollama list. I used a 4B model because my laptop has 16 GB of RAM and no GPU, and a bigger one would be impractical. The ghost covers the wait with "Hold still. Let me look." - It degrades instead of dying. If Ollama is not running, the ghost uses built-in lines and says so in the window ("voice: built-in"), rather than pretending.
What it can't do
- A screenshot is one moment, not an hour of work. Opening the right window would fool it.
- A 4B model can accept irrelevant evidence and reject real work. That is why the honour button exists.
- Speech uses the Windows speech engine, which is built in but is not an open-weight model. On other systems the ghost is silent.
- Local is not the same as secure. A screenshot can still contain sensitive things.
Try it
git clone https://github.com/newton-lebniz/haunt
cd haunt
pip install -r requirements.txt
ollama pull gemma3:4b
python haunt.py check
python haunt.py add "Finish the report" --expect "The report open with a finished conclusion" --first 0
python haunt.py run
- Code: https://github.com/newton-lebniz/haunt
- Landing page: https://newton-lebniz.github.io/haunt/
How I built it: I used AI assistants for code and debugging. I ran it, tested it and fixed the real-desktop problems on my own machine.
A ghost that watches your screen should not live on someone else's server. This one lives on mine, and it still will not let me off.
Challenge: Hacktoberfest Open-Source AI Challenge
Week 4 Theme: Ghost in the Machine
Category: Gemma
Top comments (0)