This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
In February 2015, a small office in Mons, Belgium, started keeping a diary. Once a minute, for sixteen days, it wrote down how bright it was, how much CO₂ was in the air, how warm and how damp. And once a minute, a camera took a picture, so that each line could carry one more fact: whether anyone was in the room.
That diary is the UCI Occupancy Detection dataset. For ten years it has been a beginner's machine-learning exercise: predict the last column from the others. I wanted to read it the other way round. Not "can we tell when someone is here?" but "what happens in this room when, officially, nobody is?"
The Empty Room is a séance over that log. It has three parts:
- A presence detector. TabPFN learns what a person does to a room: breath raising the CO₂, a hand on the light switch, a body warming the air. Then it listens to the 15,810 minutes the log calls empty. Every so often, it hears someone anyway.
- The Medium. For each of those presences, a local Gemma 4 speaks as the presence, in the first person, using only the numbers in the evidence.
- The Skeptic. Then the same local Gemma changes sides. With tools for the sun's position, that day's historical weather, a live web search and a way to question the detector itself, it tries to explain the ghost away. Each case ends in one of three verdicts: DEBUNKED (nobody was there), LIVING (somebody was, and the record missed them) or UNEXPLAINED.
The theme asked for something spooky, or something that honours the past. This is both: a ghost hunt where the haunted house is a dataset, and the ghosts are the traces real people left in a room eleven years ago.
I ran the séance over the strongest apparition of seven different days. The Skeptic explained six of them away: the midday sun on a Thursday, a lamp left burning at dusk on a Monday while the CO₂ of people who had just left slowly drained out of the room, a brief flash of light on an empty Sunday morning.
The seventh, it couldn't. On Friday 6 February 2015, at 07:40, half an hour before sunrise, the light in the empty room went from 0 to 260 lux, then 422. The Medium remembered it like this:
They think I was merely a whisper, an error in your perfect log. But do you feel it? I was here. […] even when the camera showed nothing but empty air.
The Skeptic took the light away from TabPFN (presence fell from 0.94 to 0.00), checked the sun (6° below the horizon) and the weather archive (no sunshine at all that hour), and ruled LIVING: a person manually turned on an artificial light source when the room was otherwise empty. In its own words, the evidence "strongly suggests a person entered to switch on a lamp, even if they were missed by the camera."
Demo
The case archive is live: empty-room-6z2k.onrender.com. All seven cases, each with its chart, the testimony and the verdict, and both voices to play.
Code
The Empty Room
A séance over a sensor log.
Case archive: https://empty-room-6z2k.onrender.com/
In February 2015 an office in Mons, Belgium recorded its light, CO₂, temperature and humidity once a minute, while a camera took a picture each minute to record whether anyone was inside. That log (UCI Occupancy Detection) has been a machine-learning benchmark for ten years. This project reads it as a ghost story.
- TabPFN learns what a person does to a room. It sees thousands of labelled minutes and works out what a body looks like in the sensors: breath raising the CO₂, a hand on the light switch, warmth. Time of day is hidden from it, so it judges presence only from the physics.
- Then it listens to the empty minutes. Each day is scored by a model that never saw that day. Minutes the log calls empty but the model calls occupied are presences…
How I Built It
The whole pipeline is about 1,000 lines of Python, run with one command:
uv run empty-room investigate --top 7
Finding the presences (TabPFN-3.5). TabPFN is a tabular foundation model: you don't train it, you show it labelled rows in context and it predicts the rest. I give it nine features per minute: the five raw sensors plus how fast CO₂, light and temperature were changing. I deliberately leave out the time of day, so it can't learn "people come in at nine". It has to judge presence from the physics alone.
To keep it honest, every day is scored by a model that never saw that day. The sixteen days are split into four folds, and each fold's context is a balanced sample of 1,000 minutes from the other three. Out of fold, it gets 99.1% balanced accuracy at telling an occupied minute from an empty one. That matters, because the interesting part is the remaining 0.9%. 254 minutes the log calls empty, TabPFN calls occupied. Grouped into runs of five minutes or more, they become 17 apparitions.
The Medium (Gemma 4, no tools). A system prompt and the evidence: the time, the day, the minute-by-minute readings, and the detector's confidence. The rules are strict: speak as the presence, quote at least three real numbers, and don't explain yourself away. The strictness is what makes the testimonies work. When the ghost says its passing left the CO₂ "falling from a high of 892.5 ppm down to 775.5 ppm", that's a real breath, decaying in a real room.
The Skeptic (Gemma 4 with tools). A small agent loop over an OpenAI-compatible endpoint, with five tools:
| Tool | What it checks |
|---|---|
sensor_window |
What the log said before and after, whether a light was switched on or off, and the CO₂ trend, plus the raw readings. Did CO₂ fall like someone had just left? |
sun_position |
The sun's elevation and azimuth over Mons at that minute (NOAA's formula, computed offline) |
historical_weather |
That day's hourly cloud cover and sunshine from the Open-Meteo archive |
counterfactual |
Re-asks TabPFN with one clue (light, breath or warmth) reset to how the empty room looks at night. If the presence vanishes without the light, the ghost was sunlight. |
web_search |
Live Google results through SerpApi: was it an ordinary working day, or a Belgian holiday? |
counterfactual is my favourite part. TabPFN isn't just the first stage of the pipeline; the agent goes back and questions it. That turns "the model flagged this" into "the model flagged this because of the light, and the sun was 6° below the horizon, so someone switched it on".
I wrote the Skeptic a few rules of thumb, the kind a careful person would use: a lamp does not switch itself on; if the log says occupied on both sides and the readings carry on unchanged, the log probably forgot someone; if the light stays on while CO₂ falls steadily, the person has left. The first version had none of them, and it called every case DEBUNKED, even one where its own report said people were "genuinely there". The other fix was giving sensor_window a computed summary. A 4B model reading a table of numbers every five minutes will sometimes misread it; a line that says "light_switched_on_at_start": true it won't. The model still makes the judgement, but the arithmetic is done for it.
The web is the least reliable witness. In five of the seven cases the Skeptic searched whether the day was a Belgian holiday, and Google mostly answered with holiday calendars for 2016, 2022 and 2027. Eleven years on, the internet barely remembers February 2015 either.
Voices (ElevenLabs). Each case is read aloud, with one voice for the presence and another for the Skeptic: two of ElevenLabs' default voices on the free plan, through the Flash model so all seven cases fit in a month's credits.
Tracing (Sentry). Every séance is one Sentry transaction. Inside it are an invoke_agent span for each Medium and Skeptic run, an execute_tool span for each tool call, and the model calls captured by Sentry's OpenAI integration. Pointed at a local server, that integration still records tokens and latency. The final run is one 19-minute trace with 138 spans, and it showed me something I hadn't expected: the slowest single step isn't a Gemma call, it's the counterfactual tool. Re-asking TabPFN "what if?" for every clue takes over a minute on the CPU.
Sentry's Agent Activity view reads the same trace as a timeline of each agent's turns. You can follow the Skeptic's investigation step by step: counterfactual, sensor window, the sun, a web search, and a chat call between each, with the exact apparition it was handed as input.
The archive. Everything lands in one static page: a chart of light, CO₂ and presence around each apparition, the testimony, the verdict and the evidence trail. The heavy work happens on my laptop, so Render only has to serve the result: a free static site, deployed from a render.yaml Blueprint in the repo, that updates on every push.
Tests and CI. The tests swap TabPFN and Gemma for tiny stubs: a "model" that thinks any light means a person, and a scripted LLM that calls tools in a fixed order. That lets GitHub Actions check the detector, the apparition grouping, the agent loop and the tool error handling on every push, without weights, a GPU or the network.
Why Does Open Innovation Matter?
Everything that thinks in this project runs on my laptop: TabPFN on the CPU, and Gemma 4 E4B on a 4 GB laptop GPU through LM Studio's OpenAI-compatible server. That changed what I could build in three ways.
It made the investigation free to repeat. A full séance over seven cases is 43 model calls and about 136,000 tokens, 28,000 of them generated. It takes about 19 minutes and costs nothing. I ran it eight times end to end while I tuned the Skeptic. On a metered API I would have run it once and kept the result, and the rules of thumb above would never have been written.
It let the agent question the model. The counterfactual tool works because TabPFN's weights are on my disk. I can ask it a hundred "what if the light had been off?" questions without a rate limit or a bill. A closed classifier behind an API would have stayed a black box. With open weights it became a witness the Skeptic can cross-examine.
It keeps the data in the room. This dataset is public, but the same pipeline works on any sensor log, like your own smart-home export. A haunted-house detector pointed at a real home is an occupancy tracker, and that data should never leave the house. Open weights mean it doesn't have to.
The Spirit of It
It landed for me on the Friday case.
At twenty to eight on the morning of 6 February 2015 it was −3 °C outside in Mons and still dark. Somebody came in first and switched on the light. The camera that recorded who was in the room caught them in exactly two frames, at 07:41 and at 07:47. In every other picture for the next ten minutes, the room was empty. Only from 07:50 did they stay in shot.
But the room itself never lost them. The lux jumped from zero. The CO₂ climbed from 434 to 460 ppm as they breathed. Every minute they spent out of frame, the sensors were still writing them down. The dataset kept that gap for eleven years, and a machine-learning benchmark has been treating those minutes as label noise ever since.
I don't know who they were. A cleaner, a researcher with a deadline, someone whose desk sat just outside the camera's view. But Día de los Muertos is about exactly this: the people the official record forgets, and the traces that remember them anyway. The ghost in this machine was never a ghost. It was a person, arriving early, in the dark, just out of shot.
Prize Categories
- Best Use of TabPFN
- Best Use of Gemma
- Best Use of SerpApi
- Best Use of ElevenLabs
- Best Use of Sentry Agent Tracing
- Best Use of GitHub Copilot
- Best Use of Render
Data: Candanedo & Feldheim (2016), UCI Machine Learning Repository, CC BY 4.0. Weather: Open-Meteo, CC BY 4.0.
Top comments (0)