CI Watchdog
An autonomous GitHub App and SaaS platform that stops wasted CI spend before it hits your invoice, and automatically diagnoses broken builds using open-weight AI.
What I Built
Every developer has that one friend or team lead who dreads the 20th of the month: the day GitHub sends the notification:
"You have used 100% of your included GitHub Actions minutes."
I built CI Watchdog for my friend and engineering teammate, who was constantly battling ballooning CI bills and jammed pipeline queues on open-source and startup projects.
The problems:
- Developers push 4 quick commits to a PR, and GitHub blindly spawns 4 redundant 15-minute test suites in parallel on expensive runners (Ubuntu, macOS, Windows).
- When a build fails after 20 minutes, developers have to dig through 30,000 lines of terminal logs just to discover a blocked database container port or a missing env variable.
The solution:
- Autonomous Waste Prevention: Detects and cancels superseded runs, runaway jobs exceeding historical p95 execution baselines, and PR workflows left running on closed branches.
-
Proactive Workflow Hygiene: Audits repository workflows for missing
concurrencyblocks and generates 1-click YAML fixes. - AI Failure Investigation: The moment a build fails, an integrated AI agent pulls the raw runner logs, redacts secrets, diagnoses the exact root cause, pinpoints the failed step, categorizes the bug, and provides copy-paste solutions.
- Human-in-the-Loop Control Plane: Includes a kill switch, dry-run observation safety rails, team approvals for cancellations, and interactive ROI spend simulators.
Demo
- 🌐 Live Dashboard: ci-watchdog.vercel.app
- 🧪 Live Demo Pull Request: Live Demo
Key Features to Explore
| Route | Feature | Description |
|---|---|---|
/app |
Control Room Overview | Live and dry-run minutes-saved counters, real-time system status indicators, and weekly savings charts. |
/app/investigations |
AI Investigations | Instant root-cause diagnostic cards generated by Gemma / Gemini 3.8 Flash, with failure hypotheses, log evidence snippets, and recommended next steps. |
/app/decisions |
Decision Engine & CSV Export | Full searchable decision history with rule and status filters, plus 1-click CSV download for engineering management reports. |
/app/approvals |
Human Approval Queue | First-N quota approval flow with 1-click Approve and Deny buttons. |
/app/digest |
Interactive ROI Calculator | Real-time savings simulator modeling monthly minute volume and runner tiers (see pricing below). |
Runner pricing used in the ROI calculator:
| Runner | Cost per minute |
|---|---|
| Ubuntu | $0.008 |
| Windows | $0.016 |
| macOS | $0.080 |
| Apple Silicon (M-series) | $0.160 |
Code
The entire codebase is open-source and structured as a high-performance TypeScript monorepo using Turborepo and pnpm:
| Path | Description |
|---|---|
apps/watchdog |
Event-driven Node.js background service deployed on Render. Connects to the GitHub App webhook stream, processes workflow states, runs the pure rule engine, and coordinates AI triage. |
apps/dashboard |
Next.js 16 (Turbopack) dashboard deployed on Vercel with Shadcn UI, Auth.js (OAuth with GitHub & Google), and real-time state synchronization. |
config/watchdog.yml |
Declarative team policies with per-repository overrides. |
How I Built It
Architecture Overview
1. Pure, Deterministic Rule Engine
The rule engine (apps/watchdog/src/rules/) was designed to be 100% pure with zero side effects. Given the current run, sibling runs, historical durations, and workflow YAML, it evaluates four rules:
| Rule | Behavior |
|---|---|
SUPERSEDED |
Cancels older runs queued on the same ref when a newer commit arrives. |
RUNAWAY_DURATION |
Compares running jobs against historical p95 execution times and alerts/cancels if a job hangs. |
MISSING_CONCURRENCY |
Scans workflow ASTs and alerts if jobs lack top-level concurrency cancellation blocks. |
STALE_ON_CLOSE |
Immediately terminates active pipelines when a PR is merged or closed. |
2. Intelligent AI Failure Triage
When a workflow concludes with a failure, CI Watchdog activates the AI investigator:
- Fetches the raw runner logs via the GitHub API.
- Runs pattern redaction to scrub tokens and sensitive credentials.
- Dispatches the condensed failure excerpt to open-weight models (Google Gemma / Gemini 3.8 Flash via OpenAI-compatible endpoints).
- Validates the output with Zod, enforcing structured JSON containing:
| Field | Description |
|---|---|
rootCause |
Explicit technical diagnosis (e.g., missing Docker service container, database port refusal). |
failedStep |
Exact bash command or step name that exited non-zero. |
category |
One of: Infrastructure, Configuration, Test, or Code. |
nextSteps |
Concrete remediation actions. |
3. Safety Rails & Human-in-the-Loop
Automated cancellation can be intimidating for developers, so I implemented strict safety rails:
- Default Dry-Run Mode: Repositories begin in observation mode so teams can review what would have been canceled without impacting active builds.
- 48-Hour Observation Window: Go-live can only be unlocked after a repository has been observed.
- First-N Live Approval Quota: The first cancellations require explicit human confirmation via dashboard or Slack buttons.
-
Protected Branch Immunity: Never cancels
main,master, orrelease/*branches. - Global Kill Switch: Instantly pauses all active cancellations across all repositories.
Why Does Open Innovation Matter?
Open innovation is what makes a tool like CI Watchdog possible without compromising developer trust:
- Data Sovereignty & Privacy: Build logs often contain internal paths, dependencies, proprietary package names, and sensitive environment details. Relying on closed, proprietary cloud AI black boxes creates data-leakage risks for enterprise codebases. Open-weight models like Gemma enable teams to run AI failure analysis locally or within their own private infrastructure.
- Deterministic Customization: Open tooling and open protocols allow developers to customize rule thresholds, inspect prompts, tune temperature, and swap backend models based on compute constraints.
- No Vendor Lock-In: Developers shouldn't need a six-figure enterprise contract just to know why their CI pipeline broke or to prevent redundant job execution. Open innovation democratizes cost control for individual creators, open-source maintainers, and startups alike.
Prize Categories
- Build for a Friend
- Open Innovation & Open-Weight AI
Built with ❤️ during Hacktoberfest. If your team is burning minutes on redundant CI runs, give CI Watchdog a spin!
Top comments (0)