What I Built
What happens when an AI task disappears from the interface, but the work underneath refuses to die?
A user clicks Cancel. The screen goes quiet. But somewhere inside the system, a worker may still be running, an acknowledgment may be delayed, or a retry may be waiting to resurrect a task that everyone believes is finished.
That is the ghost I wanted to explore.
I built GhostQueue, an open-source, local-first AI experiment that examines descriptions of interrupted or ambiguous tasks and recommends one of four states:
- Retry: The event appears recoverable.
- Review: A human decision or additional verification is needed.
- Quarantine: The input is untrusted and should be isolated.
- Complete: The event indicates successful completion.
But there is an important distinction: GhostQueue recommends actions. It never executes them.
The project explores a simple question: can we use a small, transparent AI model to interpret task failures without allowing probabilistic predictions to become uncontrolled execution decisions?
Demo
GhostQueue includes a reproducible command-line demonstration and a publicly verified GitHub Actions workflow.
View the Demo and reproduction guide
View the successful GitHub Actions run
The example scenarios include a temporary network timeout, a task involving external side effects, an untrusted instruction, and a completed job.
The automated workflow regenerates the model, runs unit tests, evaluates synthetic holdout examples, and exercises the command-line interface.
This is a CLI demonstration, not a hosted web application or a recorded video.
Code
GitHub repository: https://github.com/hippoley/ghostqueue-week4
The project is MIT-licensed and designed to run locally without a proprietary inference API.
To reproduce it:
git clone https://github.com/hippoley/ghostqueue-week4.git
cd ghostqueue-week4
python -m pip install -r requirements-train.txt
python train.py
python -m unittest discover -s tests -v
python evaluate.py
python -m ghostqueue examples/events.json
The training step generates the model weights locally. Inference subsequently runs using the Python standard library.
How I Built It
I used an open-source machine-learning stack based on scikit-learn, TF-IDF, and logistic regression.
The initial training set contains 32 synthetic event descriptions representing four categories: retry, review, quarantine, and complete.
Rather than calling a remote model endpoint, the training process exports the learned parameters into JSON. The inference component reads those parameters and performs classification locally.
The architecture separates three responsibilities:
1. Input validation
Incoming events must satisfy explicit structural and type requirements. Malformed messages, invalid boolean fields, oversized inputs, and invalid event identifiers are rejected.
2. Local AI inference
The classifier estimates which state best describes an event and returns a confidence score.
3. Deterministic policy enforcement
Safety-sensitive rules override the classifier. Untrusted input is quarantined, and events involving external side effects or required approval are routed to review.
No actual retry, deletion, payment, or other external action is performed.
The unexpected result
I also built a simple keyword-based baseline.
The project evaluates both approaches against 24 separate, author-created synthetic holdout cases.
| Measurement | Result |
|---|---|
| Local AI classifier | 21/24 — 87.5% |
| Keyword baseline | 23/24 — 95.83% |
| Deterministic policy checks | 5/5 passed |
The simpler keyword baseline outperformed the AI model.
That is not the result I originally hoped to see, but it is one of the most useful findings of the experiment.
These numbers come from a small synthetic dataset. They do not demonstrate production reliability, statistical significance, or real-world security performance.
Why Does Open Innovation Matter?
Open innovation makes it possible to investigate not only whether an AI system works, but why it fails and whether something simpler works better.
With GhostQueue, the training examples, training code, inference logic, evaluation cases, and deterministic safeguards are available for inspection.
Anyone can regenerate the model, challenge the examples, improve the evaluation, or replace the classifier without depending on a proprietary inference service.
This openness made the baseline comparison especially meaningful.
If I had reported only the AI model's 87.5% accuracy, the result might have sounded impressive. Publishing the keyword baseline reveals a different picture: on this limited dataset, additional machine learning complexity did not produce better classification.
That insight changes the engineering direction.
The next challenge is not simply to build a larger model. It is to identify the kinds of ambiguity where learned inference provides measurable value, while preserving deterministic boundaries around actions that could affect real systems.
The Spirit of It
Ghost stories usually begin with something that should be gone.
In software, the haunting can be almost invisible.
A task is cancelled. The interface reports success. Everyone moves on.
Then a delayed acknowledgment arrives. A retry awakens. An operation thought to be finished appears again.
GhostQueue does not claim to solve that entire distributed-systems problem. Instead, it makes a small part of the uncertainty observable and testable.
The moment that stayed with me was discovering that the AI classifier lost to a simple baseline.
It was a reminder that the goal of building with AI is not to put AI everywhere.
The goal is to make systems more understandable, verifiable, and trustworthy—even when the evidence tells us that the simpler solution wins.
Top comments (0)