DEV Community

Andrea Guida
Andrea Guida

Posted on

I built shadow-debugger: non-blocking AI error diagnosis for Python & FastAPI (works offline with Ollama)

#python #fastapi #opensource

shadow-debugger

License: MIT
Python 3.9+

Background error diagnosis for Python apps and web APIs.

When an exception or HTTP 500 happens, shadow-debugger captures the stack frame from your code, strips secrets from local variables and request data, and sends the context to an LLM in a background worker thread. Your app returns its normal response right away; a few seconds later you get a Markdown postmortem in ./shadow_reports/.

It works offline with Ollama (zero external dependencies) or with Gemini, OpenAI (including OpenAI-compatible endpoints like Groq, vLLM, LM Studio), and Anthropic.

What a report looks like

Given a crashing endpoint:

@app.get("/checkout/{order_id}")
async def checkout(order_id: int, discount_items: int = 0):
    unit_discount = 50 / discount_items
    return {"order_id": order_id, "discount": unit_discount}
Enter fullscreen mode Exit fullscreen mode

shadow-debugger writes shadow_reports/incident_20261018_103012_ZeroDivisionError_9f2a1c4b.md:

# Shadow Debugger Incident Report

**Timestamp:** 2026-10-18T10:30:12.418290+00:00  
**Exception:** `ZeroDivisionError`: division by zero  
**Location:** `app/routes.py:42` in `checkout`  
**Fingerprint:** `9f2a1c4b8e1032d1` (occurrences so far: 1)  

---

### 1. Root Cause Summary
`checkout()` divides `50` by `discount_items` without checking for `0`. When the query parameter is omitted or set to `0`, Python raises `ZeroDivisionError`.

### 2. Trigger Condition
`GET /checkout/1042?discount_items=0` with local state `{"order_id": 1042, "discount_items": 0}`.

### 3. Proposed Fix
```python
unit_discount = 50 / discount_items if discount_items > 0 else 0.0
```
```diff
-    unit_discount = 50 / discount_items
+    unit_discount = 50 / discount_items if discount_items > 0 else 0.0
```

### 4. Prevention Advice
Validate numeric query parameters with `Query(gt=0)` in FastAPI or guard denominators before dividing.
Enter fullscreen mode Exit fullscreen mode

Installation 

# Core package (stdlib only — works with local Ollama out of the box)
pip install shadow-debugger

# Or install with the provider / framework you use:
pip install "shadow-debugger[gemini]"
pip install "shadow-debugger[openai]"
pip install "shadow-debugger[anthropic]"
pip install "shadow-debugger[fastapi]"
pip install "shadow-debugger[flask]"
pip install "shadow-debugger[django]"
Enter fullscreen mode Exit fullscreen mode

Quick start

Local with Ollama (no API key) 

import shadow_debugger

shadow_debugger.init(
    provider="ollama",
    model="llama3.2",
    print_to_console=True,
)
Enter fullscreen mode Exit fullscreen mode

Cloud providers 

import shadow_debugger

# Uses GEMINI_API_KEY
shadow_debugger.init(provider="gemini", model="gemini-3.8-flash")

# Uses OPENAI_API_KEY (or pass base_url=... for Groq / vLLM / LM Studio)
shadow_debugger.init(provider="openai", model="gpt-5.4-nano")

# Uses ANTHROPIC_API_KEY
shadow_debugger.init(provider="anthropic", model="claude-haiku-5-5")
Enter fullscreen mode Exit fullscreen mode

Web frameworks

FastAPI / Starlette 

from fastapi import FastAPI
import shadow_debugger

shadow_debugger.init(provider="ollama", model="llama3.2")

app = FastAPI()
app.add_middleware(shadow_debugger.FastAPIWatcherMiddleware)
Enter fullscreen mode Exit fullscreen mode

Flask 

from flask import Flask
import shadow_debugger

app = Flask(__name__)
shadow_debugger.init(provider="ollama", model="llama3.2")
shadow_debugger.init_flask(app)
Enter fullscreen mode Exit fullscreen mode

init_flask hooks into got_request_exception, so normal 404/4xx responses are left untouched.

Django

In settings.py:

MIDDLEWARE = [
    "shadow_debugger.integrations.DjangoWatcherMiddleware",
    # ...
]
Enter fullscreen mode Exit fullscreen mode

Decorator & manual capture

@watch works on both regular and async def functions:

import shadow_debugger

@shadow_debugger.watch(reraise=False)
async def process_order(payload: dict):
    return payload["items"]["price"] / payload["items"]["quantity"]

# Or inside an existing try/except block:
try:
    sync_inventory()
except Exception as exc:
    shadow_debugger.capture_exception(exc, extra_metadata={"job": "inventory"})
Enter fullscreen mode Exit fullscreen mode

Privacy & what gets sent

If you use a cloud provider, the prompt includes the traceback, the 5 lines of source around the error, and the local variables of the failing function. To keep sensitive data out of prompts:

  • Variable & header names matching common secret names (password, secret, token, api_key, authorization, cookie, etc.) are replaced with [REDACTED].
  • String values matching JWTs, API keys (sk-..., AIza..., AKIA..., ghp_...), Bearer tokens, PEM private keys, and URL query tokens (?access_token=...) are scrubbed across variables, tracebacks, and exception messages.
  • Lazy objects (like Django QuerySets) are inspected via __dict__ without calling repr()/str(), so capturing an error won't trigger extra database queries.
  • You can turn off parts of the payload with send_locals=False or send_source_code=False, add your own regexes via redact_keys=[...] and redact_patterns=[...], or run everything locally with provider="ollama".
  • Proposed code patches are only checked with ast.parse() — generated code is never executed.

Configuration

All options can be passed to shadow_debugger.init(...):

Option Default Description
provider "gemini" "gemini", "ollama", "openai", "anthropic", or a Provider instance.
model None Model name. Uses the provider default if omitted.
api_key None API key (falls back to GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY).
base_url None Custom URL for Ollama (OLLAMA_HOST) or OpenAI-compatible servers.
output_dir "./shadow_reports" Folder for Markdown reports (None disables file output).
print_to_console False Print diagnoses to stderr (handy in local dev).
cooldown_seconds 300.0 Minimum seconds before re-analyzing the same error at the same line.
send_source_code True Include source lines around the crash.
send_locals True Include redacted local variables.
redact_keys () Extra regexes for variable/key names to redact.
redact_patterns () Extra regexes for secret value patterns to scrub.
request_timeout 60.0 Timeout in seconds for LLM calls.
shutdown_timeout 10.0 Seconds to wait on process exit for pending reports to finish.
max_queue_size 100 Max queued incidents before dropping new ones.
catch_uncaught True Hook sys.excepthook and threading.excepthook.

Writing a custom provider 

import shadow_debugger
from shadow_debugger import Provider, register_provider

@register_provider("my-gateway")
class MyGatewayProvider(Provider):
    default_model = "my-model"
    env_key = "MY_GATEWAY_KEY"

    def complete(self, system: str, prompt: str) -> str:
        self.check_configuration()
        # Call your LLM endpoint and return Markdown
        return "..."

shadow_debugger.init(provider="my-gateway")
Enter fullscreen mode Exit fullscreen mode

Roadmap & contributing

I'm actively looking for feedback and contributors. A few areas where PRs are very welcome:

  • [ ] Webhook / notification sinks (Slack, Discord, Telegram, GitHub Issues)
  • [ ] Additional providers (Mistral, LiteLLM, AWS Bedrock)
  • [ ] Structured JSON output option alongside Markdown reports
  • [ ] Celery / background worker task integration
  • [ ] CLI command to browse and filter local reports in ./shadow_reports

See CONTRIBUTING.md for local setup instructions.

License

MIT

Top comments (0)