DEV Community

Cover image for GhostTrace: Forensic Autopsy & Counterfactual Debugger for Autonomous AI Agents
Sazzad Hossain
Sazzad Hossain

Posted on Edited on

GhostTrace: Forensic Autopsy & Counterfactual Debugger for Autonomous AI Agents

#devchallenge #hf26challenge #ai #opensource

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

GhostTrace is an autonomous forensic autopsy platform and counterfactual debugger for AI agent workflows.

The Problem & Theme Interpretation: "Ghost in the Machine"

In modern multi-agent systems, agents don't simply crash with neat stack traces. They fail silently:

  • Hallucinating non-existent database columns or invalid tool arguments.
  • Trapped in recursive API reasoning loops while burning through millions of tokens.
  • Suffering subtle state drift, tool timeouts, and context window saturation.

When an agent fails, developers are left sifting through megabytes of noisy, unformatted JSON telemetry logs at 2 AM. These elusive, silent failures are the true "Ghosts in the Machine".

GhostTrace solves this by ingesting raw agent telemetry, building deterministic causal dependency graphs, performing grounded counterfactual root-cause deduction with Google Gemma 2, and automatically synthesizing runnable Vitest regression suites with circuit-breaker assertions to guarantee that the phantom bug never returns to production.

Demo

Watch on YouTube: GhostTrace — Forensic Autopsy & Counterfactual Debugger for AI Agents

Instructions for Judges (Testing the Application)

  1. Explore the Live Dashboard: Visit https://ghosttrace-s26u.onrender.com/. You can instantly explore the live trace waterfall, anomaly engine badges, and Gemma 2 counterfactual hypothesis cards.
  2. Try Sample Failure Traces: Click the [Upload Trace] button in the top navigation bar. You can upload any sample JSON trace from the repository's test-traces/ folder (e.g. test_trace_01_tool_crash_stripe_504.json or test_trace_02_infinite_loop_rag_query.json), or paste custom JSON.
  3. Inspect the Synthesized Vitest Suite: Scroll down to the green Anti-Regression Vitest Blueprint card to inspect or copy the runnable TypeScript test suite synthesized specifically for that failure mode.
  4. Auth0 Authentication (Optional for Testing Tenant Isolation): Click the Login button in the header to authenticate with Auth0 and test multi-tenant trace saving and private incident history under your own user session.

Key Live Features

  1. Interactive Trace Waterfall: Millisecond-precision timeline detailing token usage, tool latency, and exact error offsets.
  2. Causal Anomaly Engine: Real-time detection of infinite loops, unhandled tool errors, and context window saturation.
  3. Google Gemma 2 Hypothesis Card: Deterministic root-cause analysis with strict Event-ID grounding ([evt_104], [evt_108]), completely eliminating hallucinated diagnoses.
  4. Anti-Regression Vitest Synthesizer: One-click copy-pasteable TypeScript test suite verifying circuit breakers and schema resilience.

Code

GitHub logo diusazzad / ghosttrace

Autonomous forensic autopsy & counterfactual debugger for AI agents — powered by Mastra, Google Gemma 2, Next.js 16 & Vitest (Hacktoberfest 2026)

GhostTrace 🔍👻

Evidence-backed, deterministic forensic debugger and root cause analyzer for AI agents.
Built for the official DEV Challenge: Hacktoberfest 2026 Week 4 — "Ghost in the Machine".

Next.js 16 React 19 TypeScript 5.8 Vitest Mastra Core Google Gemma 2 Sentry Tracing MongoDB Atlas Auth0 Render

🌐 Live Production Deployment: https://ghosttrace-s26u.onrender.com/
🎥 Product Video Walkthrough: Watch on YouTube (2 min)
📝 DEV Challenge Article: Read the Official Submission

🎥 Video Walkthrough

GhostTrace Product Walkthrough

📺 Click to watch the full demo on YouTube: https://youtu.be/Lsngt5w0O3k

📸 Dashboard Overview

GhostTrace Forensic Dashboard

Figure: Interactive forensic dashboard featuring deterministic timeline reconstruction, causal anomaly detection, evidence-grounded Google Gemma 2 hypotheses, automated Vitest regression test generation, multi-tenant Auth0 login, and incident history persistence.

⚡ What is GhostTrace?

Modern AI agents are complex non-deterministic state machines. When agents fail—spinning in infinite tool loops, hallucinating invalid tool schema arguments, exceeding context windows, or corrupting session state—developers are left sifting through gigabytes of unstructured JSON telemetry.

GhostTrace is a developer-first forensic debugger that exorcises the "ghost in the machine"…

View on GitHub

How I Built It

GhostTrace is engineered strictly around open-source AI and transparent agent architecture:

flowchart TD
    A[Raw Agent Telemetry / Sentry OTEL / JSON] --> B[Mastra Core Workflow Ingestion]
    B --> C[Stage 1: Deterministic Causal Graph & Loop Anomaly Detector]
    C --> D[Stage 2: Google Gemma 2 9B Counterfactual Inference Engine]
    D --> E[Stage 3: Evidence Grounding & Anti-Hallucination Guard]
    E --> F[Stage 4: Automated Vitest Regression Test Synthesis]
    F --> G[Interactive Next.js 16 Dark-Mode Forensic Dashboard]
    G --> H[Auth0 Tenant Isolation & MongoDB Atlas Persistence]
  1. Workflow Orchestration (Mastra Core v1.80.0): A resilient 4-stage pipeline orchestrating telemetry parsing, structural failure detection, graph topology construction, and automated test synthesis.
  2. Open-Weight Counterfactual AI (Google Gemma 2 9B): Runs inference with strict prompt constraints requiring Event-ID citations, performing grounded counterfactual deduction without proprietary vendor lock-in.
  3. Observability & Agent Tracing (Sentry Agent Tracing): Live OpenTelemetry GenAI spans capturing latency, tokens, error breadcrumbs, and step-level agent actions.
  4. Data Isolation & Persistence (MongoDB Atlas): Enterprise multi-tenant database tier enforcing compound unique constraints (ownerId_1_traceId_1) and owner isolation.
  5. Zero-Trust Agent Auth (Auth0 v4): Server-side identity verification ensuring anonymous users can never access or leak sensitive agent execution history.

Why Does Open Innovation Matter?

  1. Privacy & Proprietary Trace Protection: Agent traces frequently contain API keys, confidential customer data, internal system prompts, and proprietary database schemas. Closed commercial APIs require sending raw traces to third-party cloud servers. GhostTrace allows entire organizations to run forensic investigations completely on-premises with open-weight models.
  2. Deterministic Reproducibility: Closed APIs change weights and prompt behaviors behind the scenes. Open-weight models like Gemma and open frameworks like Mastra and Temporal ensure that a forensic finding diagnosed today will yield the exact same deterministic autopsy results tomorrow.
  3. No Per-Investigation Toll: Large agent teams execute thousands of trace autopsies daily; open innovation eliminates crippling token billing for debugging infrastructure.

Prize Categories

We are entering the following categories aligned with official Hacktoberfest Week 4 tracks:

Featured Categories ($200)

  • Best Use of Render ($200): GhostTrace is fully deployed on Render with automated Next.js 16 production builds via an Infrastructure-as-Code render.yaml Blueprint.
  • Best Use of Gemma ($200): Running Google Gemma 2 (google/gemma-2-9b-it) for counterfactual root-cause deduction with strict Event-ID grounding defense.
  • Best Use of TabPFN ($200): Tabular foundation model integration for anomaly prediction and trace metric drift classification.

Partner Categories ($100)

  • Best Use of Auth0 for AI Agents ($100): Enforcing multi-tenant session verification, server-side owner scoping, and fail-closed security for agent trace histories.
  • Best Use of MongoDB Atlas ($100): Live Atlas cluster with compound indexing (ownerId_1_traceId_1), unique collision detection, and owner-isolated incident persistence.
  • Best Use of Sentry Agent Tracing ($100): Live OpenTelemetry GenAI spans (gen_ai.system, gen_ai.workflow), automated forensic breadcrumbs, and real-time distributed tracing.
  • Best Use of Mastra ($100): 4-step directed acyclic workflow (forensicWorkflow) orchestrating timeline ingestion, deterministic anomaly detection, Gemma root-cause analysis, and Vitest test synthesis.
  • Best Use of Entire ($100): Entire CLI session capture, Git checkpoint refs, and immutable commit audit trails.
  • Best Use of Ultramarine (Paper Instruments) ($100): Ultramarine-1-flash provider integration with token cost calculation and structured Zod output.
  • Best Use of Backboard ($100): Backboard Studio provider integration with offline deterministic grounding defense.
  • Best Use of Browserbase ($100): Headless browser session replay and DOM state capture for web agent failures.
  • Best Use of Temporal ($100): Temporal durable execution engine with automatic activity retries and fault recovery.
  • Best Use of Tiger Data ($100): Forensic knowledge graph and vector similarity search for historical failure patterns.
  • Best Use of ElevenLabs ($100): ElevenLabs was used to synthesize the cinematic, high-clarity voiceover narration for the official product demo video.
  • Best Use of GitHub Copilot ($100): GitHub Actions CI/CD workflows automate 162 unit and regression tests on every push and pull request.

Top comments (0)