DEV Community: Shivam Patel

Phantom Trace: an offline AI that finds the ghosts in your CI/CD logs

Shivam Patel — Mon, 26 Oct 2026 20:15:45 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine.

What I Built

Some failures never really leave a pipeline. They disappear for a week, return
under different wording, and make the next incident look new.

Phantom Trace is an offline incident-archaeology tool that finds those
recurring ghosts. You paste timestamped CI/CD events, and a local open-weight
embedding model groups failures that mean the same thing even when they share
few exact words. It then shows every source line behind the cluster.

The included fictional sample contains three incident families:

an expired TLS certificate affecting three deployment paths;
disk exhaustion reported three different ways; and
a missing deployment secret observed by two services.

One accessibility failure remains isolated. That distinction matters: the app
is meant to reveal recurrence, not force every event into a pattern.

My interpretation of Ghost in the Machine is practical rather than
supernatural. The ghosts are old operational failures still haunting a delivery
system. The machine can surface them, but it should not pretend to know more
than its evidence.

Demo

In the verified sample run, Phantom Trace analyzes nine fictional events and
returns three recurring patterns plus one isolated event. Expanding a result
reveals its timestamps, sources, and original log messages.

To run it locally:

py -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe setup_model.py
.\Start-PhantomTrace.ps1

Then open http://127.0.0.1:8766.

Model setup needs the internet once. Analysis itself does not.

Code

shivammusic / phantom-trace

Offline open-weight AI for finding recurring patterns in CI/CD incident logs.

Phantom Trace

Phantom Trace is a local incident-archaeology tool for CI/CD teams. It uses an open-weight sentence-transformer to connect historical failures that mean the same thing even when their exact wording changes, then shows every source event behind each cluster.

It was started on October 26, 2026 for the DEV Hacktoberfest Week 4 challenge Ghost in the Machine. The sample data is fictional and contains no private operational information.

Demo

Why open AI is core

Keyword searches miss x509 certificate has expired when you search for TLS renewal failure. Phantom Trace runs all-MiniLM-L6-v2 locally with ONNX Runtime to map each log message into a semantic vector. Those vectors—not a remote service—drive the recurrence graph and cluster membership.

The application does not generate diagnoses. It selects a real medoid event as the cluster title and preserves every supporting log line, so a human can audit why events were connected…

View on GitHub

Source: https://github.com/shivammusic/phantom-trace

The project is intentionally small enough to audit:

app.py contains parsing, local inference, clustering, scoring, and the HTTP server;
static/ contains the accessible browser interface;
setup_model.py downloads a pinned model revision;
tests/ exercises parsing, clustering, defaults, deterministic scoring, and failure cases; and
THIRD_PARTY_NOTICES.md records the model and prior-work attribution.

How I Built It

1. Turn each message into a local semantic vector

Phantom Trace uses
sentence-transformers/all-MiniLM-L6-v2
through ONNX Runtime. The tokenizer truncates messages to 256 tokens. The
server performs attention-mask-aware mean pooling and L2 normalization, so
cosine similarity becomes a dot product between unit vectors.

That open-weight model is the functional core. A keyword search for disk full would miss no space left on device while extracting image layers.
Their embeddings can still land close enough to connect them.

2. Build an evidence graph, not an invented diagnosis

Each event is a node. A pair becomes connected when its cosine similarity meets
the selected threshold. Connected components become incident clusters.

For the title, Phantom Trace chooses the medoid: the real event whose vector is
most central to the cluster. It does not ask a language model to generate a
root cause. Every displayed claim can be traced to an input line.

The haunting score is also deliberately ordinary. It combines recurrence,
source breadth, and recency. It is a prioritization heuristic, not an AI
confidence score.

3. Calibrate with a real model run

My first default threshold was 0.72. The UI looked polished, but a real model
integration run exposed a problem: it found only one of the three intended
recurring families.

I tested the same nine-event fixture across several thresholds. At 0.55, the
model produced the intended 3, 3, 2, 1 cluster sizes: three recurring groups
and one isolated event. I changed the default, documented why, and added a
regression test for the setting.

The slider remains available because one threshold cannot fit every log corpus.
Teams can raise it when they prefer fewer, tighter matches.

4. Keep the boundary small and inspectable

The local server accepts only 2–200 non-empty events in this format:

ISO timestamp | source | message

It bounds source and message lengths, rejects malformed input, listens on
127.0.0.1, and makes no analytics or inference API calls. Users should still
redact secrets before pasting logs; local processing is not an excuse to put
credentials into diagnostic tools.

Validation

I ran:

6 unit tests: all passed;
JavaScript syntax validation: passed;
a real local MiniLM integration run: 9 events, 3 recurring clusters, sizes 3, 3, 2, 1; and
an end-to-end browser run in Microsoft Edge, including expansion of the original evidence lines.

Why Does Open Innovation Matter?

Operational logs can contain customer names, internal hostnames, repository
paths, incident details, and secrets that should have been redacted but were
not. Shipping all of that to a closed embedding endpoint just to compare two
sentences is a poor default.

An open-weight model changes the trust boundary:

the log text stays on the operator's machine;
inference still works after the model is downloaded and the network is off;
there is no per-event API charge or quota;
the exact model revision is pinned and replaceable; and
the pooling, similarity, grouping, and scoring logic are all inspectable.

Open also makes limitations easier to discuss honestly. Semantic similarity is
not root-cause analysis. A connected cluster can be wrong, and a real recurring
incident can be missed. Phantom Trace keeps the original evidence visible so a
human can challenge the grouping instead of trusting an opaque diagnosis.

For this project, open innovation is not a licensing footnote. It is what makes
private, offline incident archaeology possible.

The Spirit of It

The moment the project landed was not an animation or a jump scare. It was the
first real-model run after calibration: three differently worded failure
families emerged from nine lines, while an unrelated accessibility failure
stayed alone.

That is the kind of ghost I wanted to catch—an old failure returning in a new
sentence, with enough evidence left behind for someone to finally recognize it.

AI Assistance Disclosure

I used OpenAI Codex to help implement, test, review, and draft documentation for
this project. I reviewed the resulting code and prose, ran the stated tests and
real-model checks locally, and corrected the similarity default after the
integration result contradicted the initial assumption. The sample data is
fictional. No private production logs were used.

Steady Inbox: A Local MiniLM Assistant That Groups Messages Without Losing the Originals

Shivam Patel — Sat, 24 Oct 2026 16:28:36 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.

What I Built

Steady Inbox is a small local-AI tool for the repetitive part of inbox review: noticing when several messages are really about the same thing. You paste one message per line, and it proposes semantic groups while keeping every original visible and exportable.

The project follows the challenge's “Slow and Steady” theme by reducing a pile of repeated reminders to a shorter review queue. It does not delete, send, archive, or act on a message. The person reviewing the results stays in control.

In my synthetic demonstration, 9 messages became 6 review groups and all 9 originals remained recoverable. That is a functional test result, not a claim about production-scale accuracy or guaranteed time savings.

Demo

Watch the verified 64-second browser demonstration.

The reproducible demonstration uses only synthetic messages: two pull-request reminders, two milk reminders, two meeting-time notes, two database-backup reminders, and one Wi-Fi notice. It shows the empty state, input validation, local grouping, an expanded two-message group, the reviewed control, and Markdown export availability.

Local reproduction:

py -3.12-64 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.lock
.\.venv\Scripts\python.exe setup_model.py
.\.venv\Scripts\python.exe app.py --port 8797

Then open http://127.0.0.1:8797/ and choose Try an example followed by Make a steady digest.

Code

github.com/shivammusic/steady-inbox

The repository will include the application, pinned direct and tested dependency lists, model manifest and hashes, automated tests, MIT license, and third-party notices. Model weights are intentionally not committed.

How I Built It

The application is a Python 3.12 HTTP service with a dependency-free browser interface. It uses:

sentence-transformers/all-MiniLM-L6-v2 at pinned revision 1110a243fdf4706b3f48f1d95db1a4f5529b4d41;
Hugging Face Tokenizers 0.22.1 for the pinned tokenizer;
ONNX Runtime 1.23.2 for CPU inference;
NumPy 2.3.3 for attention-mask-aware mean pooling, L2 normalization, and cosine similarity.

Each non-empty line receives a 384-dimensional embedding. The grouping rule is conservative complete-link clustering: every pair of messages in a proposed group must meet the chosen cosine-similarity threshold. That prevents A and C from being joined only because each resembles B. A group representative is one of the original messages—the item with the best average within-group similarity—rather than a generated summary.

The model input is capped at 256 tokens, but the application keeps the complete source text and flags truncation. Inputs are capped at 100 messages and 1,500 characters per message.

I verified the implementation with:

.\.venv\Scripts\python.exe -m unittest -v
node --check static\app.js

All 10 Python tests passed on Windows 11. They cover real model inference, normalized 384-dimensional vectors, semantic paraphrases, unrelated notes, duplicate inputs, complete source preservation, truncation reporting, validation, and protection against similarity chaining. The JavaScript syntax check also passed.

Why Does Open Innovation Matter?

The open-weight model is the feature, not decoration around a hosted API. Once its pinned files have been downloaded and verified, inference runs on the user's CPU. That gives someone a meaningful choice about where their pasted text is processed and lets a developer inspect or replace the tokenizer, weights, pooling logic, threshold, or grouping strategy.

Steady Inbox does not need an API key, usage billing, or a cloud inference service. Its interface loads no third-party scripts, and its server binds to 127.0.0.1. Notes are held in browser memory and the local request; the app does not automatically write them to disk. Those choices do not make it a secure-erasure tool—browser state and operating-system memory still exist—and an initial network connection is required to download the pinned model files.

The Lazy Payoff

The measured synthetic run reduced 9 lines to 6 groups for review while preserving all 9 originals. Three repeated items were grouped. I did not run a timed human baseline, so I am not claiming a number of minutes saved.

The smaller payoff may be more important: the app proposes structure but never hides the evidence. A reviewer can expand a group, see exactly which original lines produced it, mark it reviewed, and export a Markdown digest.