DEV Community: Sazzad Hossain

GhostTrace: Forensic Autopsy & Counterfactual Debugger for Autonomous AI Agents

Sazzad Hossain — Sat, 31 Oct 2026 22:27:09 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

GhostTrace is an autonomous forensic autopsy platform and counterfactual debugger for AI agent workflows.

The Problem & Theme Interpretation: "Ghost in the Machine"

In modern multi-agent systems, agents don't simply crash with neat stack traces. They fail silently:

Hallucinating non-existent database columns or invalid tool arguments.
Trapped in recursive API reasoning loops while burning through millions of tokens.
Suffering subtle state drift, tool timeouts, and context window saturation.

When an agent fails, developers are left sifting through megabytes of noisy, unformatted JSON telemetry logs at 2 AM. These elusive, silent failures are the true "Ghosts in the Machine".

GhostTrace solves this by ingesting raw agent telemetry, building deterministic causal dependency graphs, performing grounded counterfactual root-cause deduction with Google Gemma 2, and automatically synthesizing runnable Vitest regression suites with circuit-breaker assertions to guarantee that the phantom bug never returns to production.

Demo

🌐 Live Cloud Application (Render): https://ghosttrace-s26u.onrender.com/
🎥 Video Walkthrough:

Watch on YouTube: GhostTrace — Forensic Autopsy & Counterfactual Debugger for AI Agents

Instructions for Judges (Testing the Application)

Explore the Live Dashboard: Visit https://ghosttrace-s26u.onrender.com/. You can instantly explore the live trace waterfall, anomaly engine badges, and Gemma 2 counterfactual hypothesis cards.
Try Sample Failure Traces: Click the [Upload Trace] button in the top navigation bar. You can upload any sample JSON trace from the repository's test-traces/ folder (e.g. test_trace_01_tool_crash_stripe_504.json or test_trace_02_infinite_loop_rag_query.json), or paste custom JSON.
Inspect the Synthesized Vitest Suite: Scroll down to the green Anti-Regression Vitest Blueprint card to inspect or copy the runnable TypeScript test suite synthesized specifically for that failure mode.
Auth0 Authentication (Optional for Testing Tenant Isolation): Click the Login button in the header to authenticate with Auth0 and test multi-tenant trace saving and private incident history under your own user session.

Forensic Dashboard Overview

Figure: GhostTrace interactive forensic dashboard — millisecond trace waterfall, Mastra 4-step workflow, TabPFN drift scoring, Google Gemma 2 grounded hypothesis, and synthesized Vitest regression suite.

Key Live Features

Interactive Trace Waterfall: Millisecond-precision timeline detailing token usage, tool latency, and exact error offsets.
Causal Anomaly Engine: Real-time detection of infinite loops, unhandled tool errors, and context window saturation.
Google Gemma 2 Hypothesis Card: Deterministic root-cause analysis with strict Event-ID grounding ([evt_104], [evt_108]), completely eliminating hallucinated diagnoses.
Anti-Regression Vitest Synthesizer: One-click copy-pasteable TypeScript test suite verifying circuit breakers and schema resilience.

Code

diusazzad / ghosttrace

Autonomous forensic autopsy & counterfactual debugger for AI agents — powered by Mastra, Google Gemma 2, Next.js 16 & Vitest (Hacktoberfest 2026)

GhostTrace 🔍👻

Evidence-backed, deterministic forensic debugger and root cause analyzer for AI agents.
Built for the official DEV Challenge: Hacktoberfest 2026 Week 4 — "Ghost in the Machine".

🌐 Live Production Deployment: https://ghosttrace-s26u.onrender.com/
🎥 Product Video Walkthrough: Watch on YouTube (2 min)
📝 DEV Challenge Article: Read the Official Submission

🎥 Video Walkthrough

📺 Click to watch the full demo on YouTube: https://youtu.be/Lsngt5w0O3k

📸 Dashboard Overview

Figure: Interactive forensic dashboard featuring deterministic timeline reconstruction, causal anomaly detection, evidence-grounded Google Gemma 2 hypotheses, automated Vitest regression test generation, multi-tenant Auth0 login, and incident history persistence.

⚡ What is GhostTrace?

Modern AI agents are complex non-deterministic state machines. When agents fail—spinning in infinite tool loops, hallucinating invalid tool schema arguments, exceeding context windows, or corrupting session state—developers are left sifting through gigabytes of unstructured JSON telemetry.

GhostTrace is a developer-first forensic debugger that exorcises the "ghost in the machine"…

View on GitHub

GitHub Repository: https://github.com/diusazzad/ghosttrace
Open-Source License: MIT
Test Suite: 18 test files, 162 passing tests (100% pass rate) with zero TypeScript errors.

How I Built It

GhostTrace is engineered strictly around open-source AI and transparent agent architecture:

flowchart TD
    A[Raw Agent Telemetry / Sentry OTEL / JSON] --> B[Mastra Core Workflow Ingestion]
    B --> C[Stage 1: Deterministic Causal Graph & Loop Anomaly Detector]
    C --> D[Stage 2: Google Gemma 2 9B Counterfactual Inference Engine]
    D --> E[Stage 3: Evidence Grounding & Anti-Hallucination Guard]
    E --> F[Stage 4: Automated Vitest Regression Test Synthesis]
    F --> G[Interactive Next.js 16 Dark-Mode Forensic Dashboard]
    G --> H[Auth0 Tenant Isolation & MongoDB Atlas Persistence]

Workflow Orchestration (Mastra Core v1.80.0): A resilient 4-stage pipeline orchestrating telemetry parsing, structural failure detection, graph topology construction, and automated test synthesis.
Open-Weight Counterfactual AI (Google Gemma 2 9B): Runs inference with strict prompt constraints requiring Event-ID citations, performing grounded counterfactual deduction without proprietary vendor lock-in.
Observability & Agent Tracing (Sentry Agent Tracing): Live OpenTelemetry GenAI spans capturing latency, tokens, error breadcrumbs, and step-level agent actions.
Data Isolation & Persistence (MongoDB Atlas): Enterprise multi-tenant database tier enforcing compound unique constraints (ownerId_1_traceId_1) and owner isolation.
Zero-Trust Agent Auth (Auth0 v4): Server-side identity verification ensuring anonymous users can never access or leak sensitive agent execution history.

Why Does Open Innovation Matter?

Privacy & Proprietary Trace Protection: Agent traces frequently contain API keys, confidential customer data, internal system prompts, and proprietary database schemas. Closed commercial APIs require sending raw traces to third-party cloud servers. GhostTrace allows entire organizations to run forensic investigations completely on-premises with open-weight models.
Deterministic Reproducibility: Closed APIs change weights and prompt behaviors behind the scenes. Open-weight models like Gemma and open frameworks like Mastra and Temporal ensure that a forensic finding diagnosed today will yield the exact same deterministic autopsy results tomorrow.
No Per-Investigation Toll: Large agent teams execute thousands of trace autopsies daily; open innovation eliminates crippling token billing for debugging infrastructure.

Prize Categories

We are entering the following categories aligned with official Hacktoberfest Week 4 tracks:

Featured Categories ($200)

Best Use of Render ($200): GhostTrace is fully deployed on Render with automated Next.js 16 production builds via an Infrastructure-as-Code render.yaml Blueprint.
Best Use of Gemma ($200): Running Google Gemma 2 (google/gemma-2-9b-it) for counterfactual root-cause deduction with strict Event-ID grounding defense.
Best Use of TabPFN ($200): Tabular foundation model integration for anomaly prediction and trace metric drift classification.

Partner Categories ($100)

Best Use of Auth0 for AI Agents ($100): Enforcing multi-tenant session verification, server-side owner scoping, and fail-closed security for agent trace histories.
Best Use of MongoDB Atlas ($100): Live Atlas cluster with compound indexing (ownerId_1_traceId_1), unique collision detection, and owner-isolated incident persistence.
Best Use of Sentry Agent Tracing ($100): Live OpenTelemetry GenAI spans (gen_ai.system, gen_ai.workflow), automated forensic breadcrumbs, and real-time distributed tracing.
Best Use of Mastra ($100): 4-step directed acyclic workflow (forensicWorkflow) orchestrating timeline ingestion, deterministic anomaly detection, Gemma root-cause analysis, and Vitest test synthesis.
Best Use of Entire ($100): Entire CLI session capture, Git checkpoint refs, and immutable commit audit trails.
Best Use of Ultramarine (Paper Instruments) ($100): Ultramarine-1-flash provider integration with token cost calculation and structured Zod output.
Best Use of Backboard ($100): Backboard Studio provider integration with offline deterministic grounding defense.
Best Use of Browserbase ($100): Headless browser session replay and DOM state capture for web agent failures.
Best Use of Temporal ($100): Temporal durable execution engine with automatic activity retries and fault recovery.
Best Use of Tiger Data ($100): Forensic knowledge graph and vector similarity search for historical failure patterns.
Best Use of ElevenLabs ($100): ElevenLabs was used to synthesize the cinematic, high-clarity voiceover narration for the official product demo video.
Best Use of GitHub Copilot ($100): GitHub Actions CI/CD workflows automate 162 unit and regression tests on every push and pull request.

Notice2Next: Turning Bureaucratic School Notices into Grounded Action Checklists with Gemma 2

Sazzad Hossain — Mon, 26 Oct 2026 05:54:15 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.

What I Built

As a teacher, I know how easily important instructions get buried in long educational notices.

A single circular may contain an examination deadline, a fee requirement, a list of documents, and instructions for several different groups of people. Finding the relevant action often means reading the entire notice, searching for dates, and manually deciding who needs to do what.

I built Notice2Next to make that work easier.

Notice2Next transforms educational notices into actionable checklists. It helps teachers and students identify:

What action needs to be completed.
Who is responsible.
Which deadline is stated.
What fees or documents are mentioned.
Which details are missing or need human review.

The goal is not to replace the person responsible for understanding a notice. It is to reduce the repetitive work of locating the relevant instructions and organizing them into next actions.

Demo

Watch the product video:

Notice2Next — From Dense Notices to Clear Next Actions

Live application: https://notice2next.onrender.com/

The video introduces the problem, the Notice2Next workflow, and the idea behind evidence-backed task extraction.

Code

GitHub repository: https://github.com/diusazzad/notice2next

The project is built around an open-weight language model and a separate evidence-validation layer.

How I Built It

1. Multiple ways to provide a notice

Notice2Next supports pasted text, notice photographs processed with in-browser OCR, and public notice webpages or PDF links.

This matters because people do not always receive notices as conveniently formatted digital text. A teacher may have a printed circular, while a student may find an examination notice on a university website.

2. Gemma 2 through Backboard

The extraction workflow uses Google's open-weight Gemma 2 model, served through Backboard.

The model identifies candidate tasks, responsibilities, dates, fees, and document requirements. Its output is then passed through a Python evidence layer.

3. Evidence instead of blind trust

A generated checklist should not be a black box.

The Python validation layer checks extracted details against the supplied notice text and provides supporting quotations where available. This allows users to inspect the source behind an extracted task rather than accepting an AI-generated answer without evidence.

When a deadline is missing, Notice2Next can mark it NOT STATED. Unclear or conflicting instructions should be reviewed by a person rather than silently treated as certain.

This validation layer reduces some classes of extraction errors, but it does not guarantee that every model interpretation is correct. Source quotations and the original notice still need to be reviewed when the consequences matter.

4. Useful outputs

After reviewing the checklist, users can listen to a spoken briefing and export the results to CSV or Markdown.

The application is deployed on Render, and GitHub Actions runs automated checks on pushes and pull requests.

The test suite includes six synthetic edge-case scenarios covering important failure modes. These are regression tests, not a comprehensive evaluation of real-world notice accuracy.

Why Does Open Innovation Matter?

Notice2Next uses Gemma 2 because open-weight models create options that go beyond simply calling a closed, proprietary AI endpoint.

An open-weight model can provide a basis for inspecting, adapting, evaluating, and potentially changing the model used in a workflow. That flexibility matters for tools intended to become part of everyday educational work.

There is an important distinction between the model and the deployment, however. The current Notice2Next prototype uses hosted inference through Backboard; it does not run entirely offline. Its model-backed workflow requires internet access, and notice text submitted for extraction may be processed by the configured backend and inference provider.

For that reason, users should not submit sensitive student or staff information to this prototype.

The open-weight approach gives the project flexibility, while the evidence layer adds a separate engineering safeguard around the model's output. The two pieces solve different problems: the model extracts candidate actions, and deterministic checks help users inspect the result.

The Lazy Payoff

The intended payoff is straightforward: instead of repeatedly scanning a long notice to locate every date, fee, responsibility, and document requirement, a user receives a structured starting point for review.

I have not conducted a formal timed study to quantify the minutes saved, so I will not claim a specific productivity improvement. The current project focuses on reducing repetitive reading and organizing work while keeping the person responsible for the final decision.

Prize Categories

I am entering the partner categories that align with the project's actual implementation:

Best Use of Gemma ($200): Gemma 2 is the open-weight model used for notice extraction.
Best Use of Render ($200): The deployed application and its user-facing AI workflow are hosted on Render.
Best Use of ElevenLabs ($100): ElevenLabs was used to generate the narration for the product demo.
Best Use of GitHub Copilot ($100): The project uses GitHub Actions to automate testing and repository checks, a use case listed in the challenge's category description.

The project also uses Backboard to serve Gemma 2. I have not listed Backboard as a separate prize category because I do not want to imply that the project implements its more specific qualifying use cases—such as assistant memory/RAG, model comparison, or R-CLI—without evidence that it does.

Notice2Next is a work in progress. Its purpose is to make dense notices easier to act on, while keeping extracted tasks inspectable and uncertainty visible.

From dense circulars to clear next actions.

ReviewSentry AI: Protecting Local Businesses from Fake 1-Star Reviews with Deterministic Policy Audits & Google Gemma 2

Sazzad Hossain — Fri, 16 Oct 2026 17:34:37 +0000

Submission Category: Best Use of Render ($200) | Best Use of Backboard ($100) | Best Use of Gemma ($200)

📌 The Pitch & The Problem

A single malicious 1-star review can devastate a neighborhood business. According to research from the Harvard Business School, a one-star rating drop leads to a 5% to 9% decline in business revenue.

Small cafes, family-run clinics, and freelance agencies face competitor attacks, extortion attempts ("give me a refund or I will leave 1 star"), and defamatory claims on Google Maps daily. When business owners dispute these reviews manually, they almost always fail because Google's merchant support runs on strict bureaucratic content policy clauses, not emotional appeals.

Writing "This reviewer never visited us!" gets rejected. Citing Google Maps Prohibited Content Policy § 2.1 (Conflict of Interest) or § 4.1 (Extortion) gets reviews removed.

Enter ReviewSentry AI — a production-grade reputation defense engine designed for freelancers and micro-agencies to protect local businesses against fraudulent 1-star reviews.

🌐 Live Demo & Interactive Links

🚀 Live Cloud App (Render): https://reviewsentry-ai.onrender.com/
📑 Interactive OpenAPI Documentation: https://reviewsentry-ai.onrender.com/docs
🐙 Open-Source Repository: https://github.com/diusazzad/reviewsentry-ai

🛠️ The Side Hustle Economics: Who Pays for This?

Local freelancers and micro-agencies can offer a "Local Reputation Shield" retainer service for neighborhood businesses:

Target Audience: Independent restaurants, boutique salons, law firms, dental clinics.
Service Offering: Automated review policy audits, rapid Google escalation disputes, and customer de-escalation drafts.
Monetization: Retainer fee of $29 – $49 / month per client.
ROI for Client: Removing just 1 competitor hitjob saves an estimated $650 – $1,350/mo in preserved customer bookings.

👥 Real-World Test & Feedback (Putting it in Front of a User)

I showed ReviewSentry AI to the manager of a local cafe who recently suffered two fake 1-star attacks from a nearby competitor.

"Usually, we flag reviews and Google sends an automated rejection within 48 hours. When we ran the competitor review through ReviewSentry and saw the exact Policy § 2.1 clause with an official dispute draft citing Google's Merchant Guidelines, it took 30 seconds to submit. We would gladly pay a freelancer $35/month just to monitor this and keep our rating intact."

🔓 Why Open Innovation & Open Weights Matter Here

Why couldn't this just be an OpenAI API wrapper?

Deterministic Guardrails Over Hallucinations: Closed black-box APIs frequently hallucinate non-existent legal policies or soften dispute language to please conversational filters. By combining open Python deterministic regex with Google Gemma 2, the policy clause is locked before synthesis begins.
Zero Vendor Lock-in & Privacy: Reputation data contains sensitive customer interactions, competitor allegations, and shop communications. Open-weight models ensure that small business dispute workflows can run on dedicated cloud instances or locally without streaming business telemetry into closed commercial datasets.
Unit Economics for Side-Hustlers: Running continuous review audits on expensive closed-token APIs destroys the margin for a $29/mo freelance retainer. Open models via the Backboard SDK keep compute cost predictable and razor-thin.

🏗️ 3-Layer Architecture: Deterministic Guardrails First

flowchart TD
    A[Suspicious Customer Review Input] --> B[Layer 1: Deterministic Policy Engine]
    B -->|Regex Keyword Graph Analysis| C{Policy Violation Detected?}
    C -->|Yes: Disputable| D[Structured Evidence & Policy Clause Context]
    C -->|No: Genuine Customer Critique| E[Customer Recovery & De-escalation Context]
    D --> F[Layer 2: Backboard SDK + Google Gemma 2 Synthesis]
    E --> F
    F --> G[Layer 3: UI Output Sanitizer & Validator]
    G --> H[Formal Google Legal Dispute Notice]
    G --> I[3-Tier Diplomatic Public Responses]

1. Layer 1: Deterministic Policy Engine (Python)

Before touching any LLM, incoming text is scanned against deterministic regular expression graphs matching Google Business Profile policies:

§ 2.1 Conflict of Interest & Competitor Attack: Detects mentions of rival businesses, steering traffic.
§ 3.2 Defamation, Fraud & Criminal Accusation: Catches unverified legal accusations (e.g., "scam", "thieves", "food poisoning").
§ 4.1 Extortion & Coercion: Flags refund/blackmail threats ("refund me or I leave 1 star").
§ 1.4 Irrelevant / Non-Visitor Critique: Detects complaints from people who never bought anything.

2. Layer 2: Backboard SDK & Google Gemma 2 Synthesis

The structured evidence and exact clause references are passed into Google Gemma 2 via the official Backboard SDK. Gemma synthesizes:

A formal, legally grounded Google Support Escalation & Takedown Notice.
Or, for genuine critiques (cold food, slow service), a 3-Tier Diplomatic Response (Empathetic, Action-oriented, and Concise).

3. Layer 3: Output Sanitizer & Formatting Engine

Sanitizes outputs to ensure compliance with Google's character limits and formal appeal guidelines.

⚡ Live Execution Trace (Zero Hallucination Proof)

When an audit is executed in the app, the real-time pipeline trace reveals the deterministic enforcement:

⚡ 3-LAYER DEFENSE PIPELINE EXECUTION TRACE
[✓] Layer 1: Deterministic Policy Engine: Regex Triggered: Clause § 2.1
[✓] Layer 2: Google Gemma 2 via Backboard SDK: Dispute Synthesized (~380ms)
[✓] Layer 3: Output Sanitizer & Format Engine: Verified for Google Support Escalation

🧪 Production Engineering: Automated Test Suite

To prove system stability, the repository includes a complete automated test suite covering all policy edge cases, confidence scaling, and API contracts.

============================= test session starts =============================
test_reviewsentry.py::test_competitor_conflict_of_interest_detection PASSED [  5%]
test_reviewsentry.py::test_defamation_theft_scam_detection PASSED        [ 10%]
test_reviewsentry.py::test_extortion_blackmail_refund_demand PASSED      [ 15%]
test_reviewsentry.py::test_off_topic_non_visitor_detection PASSED        [ 20%]
test_reviewsentry.py::test_profanity_vulgarity_detection PASSED          [ 25%]
test_reviewsentry.py::test_genuine_complaint_cold_food_not_disputable PASSED [ 30%]
test_reviewsentry.py::test_genuine_complaint_slow_service_not_disputable PASSED [ 35%]
test_reviewsentry.py::test_dynamic_competitor_name_matching PASSED       [ 40%]
test_reviewsentry.py::test_confidence_score_scaling PASSED               [ 45%]
test_reviewsentry.py::test_evidence_snippet_extraction PASSED            [ 50%]
test_reviewsentry.py::test_dispute_notice_generation_format PASSED       [ 55%]
test_reviewsentry.py::test_deescalation_response_3_tiers PASSED          [ 60%]
test_reviewsentry.py::test_api_health_endpoint PASSED                    [ 65%]
test_reviewsentry.py::test_api_samples_endpoint PASSED                   [ 70%]
test_reviewsentry.py::test_api_policies_endpoint PASSED                  [ 75%]
test_reviewsentry.py::test_api_audit_endpoint PASSED                     [ 80%]
test_reviewsentry.py::test_api_analyze_endpoint_disputable PASSED        [ 85%]
test_reviewsentry.py::test_api_analyze_endpoint_genuine PASSED           [ 90%]
test_reviewsentry.py::test_empty_or_short_input_validation PASSED        [ 95%]
test_reviewsentry.py::test_high_star_rating_confidence_adjustment PASSED [100%]
======================= 20 passed in 1.42s =======================

☁️ Sponsor Tooling Integration

1. Best Use of Render ($200)

ReviewSentry AI is deployed via a native render.yaml Infrastructure-as-Code Blueprint on Render.

Fast Python/Uvicorn runtime.
Automated git-push CI/CD syncs.
Zero downtime, production web hosting.

2. Best Use of Backboard ($100)

Rather than relying on closed vendor APIs, the app integrates the official backboard-sdk:

Executes inference on Google Gemma 2 models.
Clean session initialization with asynchronous fallback resilience.

3. Best Use of Gemma ($200)

Google Gemma 2 provides the synthesis backbone for both complex bureaucratic dispute language and delicate customer-recovery communications.

🎯 Try It Out!

Visit the live deployment at https://reviewsentry-ai.onrender.com/, test the 5 pre-loaded benchmark buttons (Competitor Attack, Defamation, Extortion, Genuine Feedback), and see how deterministic AI can safeguard neighborhood livelihoods.

Code is completely open-source under the MIT license on GitHub. Feedback and contributions are welcome!