AiWatchDog, monitor your agents, increase productivity

Kyalo460 — Sat, 03 Oct 2026 01:50:17 +0000

I Built an Open Source Watchdog for AI Coding Agents
Live demo → https://aiwatchdog.vercel.app/ · Open source (MIT)

The problem
I run a lot of AI coding agents — Kilo, Claude Code, Codex, Aider, my own scripts. And I kept hitting the same wall:

I start a task, walk away, come back 20 minutes later, and stare at a terminal with no idea whether the agent is crushing it, waiting for me, or wedged in an infinite retry loop burning tokens.

So I built the thing I kept wishing existed.

AI Watchdog — real-time visibility and intervention for AI coding agents. It ingests everything your agent does, decides with a multi-signal confidence model whether it's working / waiting / genuinely stuck, and notifies you only when you actually matter to the outcome.

Why "stuck detection" is the hard part
The naive rule — no output for 5 minutes → STUCK — is wrong, and I refused to ship it. Agents legitimately go silent while compiling, installing dependencies, running Docker, or grinding through a long test suite. Alert on that and you'll mute the tool within a day.

So Watchdog runs a weighted ensemble of independent detectors, each producing an explainable signal:

Signal Weight Suppressed by
No heartbeat 20 explicit waiting_for_user
No tool activity 20 known long phases (build/test/docker)
Repeated similar errors 25 one-off failures
Repeated commands 15 parameterized commands
No filesystem changes 10 non-coding executions
Process looks idle 10 returns null if processes aren't visible

Confidence is the clamped sum of triggered weights — and every contributing weight is traceable, so the UI can always answer "why does it think you're stuck?"

Here's what I'm proudest of: the arithmetic works in your favor. Total silence scores 55 — five points below the 60 threshold. A coding execution with no file changes hits 65 and crosses. Silence is not evidence. That was a design rule, not a tuning accident.

What's in it
Live Command Center — every execution streaming over WebSocket
Trace view — the full event tree, so you see the exact moment it derailed
One command connects any agent: watchdog wrap -- kilo run "fix the failing test" Kilo, Claude Code, Codex, Aider, OpenCode, npm test, your own script. stdin passes through, exit code propagates, heartbeats every 15s.
TypeScript + Python SDKs — five lines to instrument an existing agent
VS Code extension — terminal commands and file changes
A stuck-agent simulator with 5 modes, so you can prove the detector before trusting it
Multi-tenant by construction — organizationId is required in every repo method, so a method that forgets it doesn't compile
Fail-closed redaction — if the redaction module can't load, ingest throws rather than persisting raw secrets
~1,500 tests, nothing mocked in e2e — real API process, real Postgres, real WebSocket, real child CLI
The honest hard parts
I shipped a cross-tenant read oracle and then found it. A global UNIQUE(dedupe_key) with an unscoped lookup meant anyone guessing stuck: could read another tenant's alert. Rescoped to (organization_id, dedupe_key).

My partition helper could delete the dev database. pg_class matched across all schemas, so a parallel test schema made a test drop production partitions. I observed this rather than theorized about it. Now pinned to current_schema().

State writes weren't compare-and-set — a read-then-write race. Now UPDATE ... WHERE id=$1 AND state=$2.

Deliberate deviations: no Next.js (SSR buys nothing for an auth-gated realtime SPA), no ORM (hand-written SQL, deterministic, full control of monthly RANGE partitioning + CAS), and the detection engine is pure — no DB, no network, no process.env — which is exactly what makes it exhaustively testable.

The load-bearing detail nobody sees: persisting hysteresis state is the write that matters. Skip it and STUCK detection silently never fires.

What's next
Extracting the sweeper to a real BullMQ worker with advisory locks, so multiple API replicas stop double-sweeping
Turning on real adapter capabilities — supportsPauseResume and supportsProcessMetrics are false for every adapter today. I'd rather ship an honest capability matrix than pretend
A real observability loop for editor-based agents — the VS Code extension can't see chat traffic inside another extension (Kilo, Cline, Roo); no API exposes it, so the localhost bridge plus terminal/file signals is the honest ceiling right now
SSO/MFA, audit logs, richer notification rules, deeper CI integrations
Why open source
The interesting part — the detection methodology, the weight table, the suppression rules, the hysteresis design — is more valuable shared than defended.

LLM tracing tools see model calls but not the shell, file, and git activity where agents actually get stuck. APM tools don't model agent execution at all. Agent platforms are closed and single-vendor. Nobody owns the neutral, execution-centric layer. I'd like this to be it.

→ Try it at https://aiwatchdog.vercel.app/

If you've ever lost 20 minutes wondering whether your agent was alive, I'd love to hear about it. And if your agent does something Watchdog doesn't understand yet, tell me — those reports are literally the roadmap.

DEV Community: Kyalo460

AiWatchDog, monitor your agents, increase productivity