DEV Community

Suzanne Mok
Suzanne Mok

Posted on

7 Bugs in 34 Days: What Our AI Agents Caught While Nobody Was Watching

We just spent 34 days running 9 AI agents on 2 CPU cores and 3.6GB RAM in a real gym.

Nobody saw most of it. That's the point.

The system caught and auto-recovered from 7 infrastructure bugs — without an SRE rotation.

Here's exactly what happened inside.


Bug 1: Memory Creep — Memory climbed from 6.8GB to 13.4GB over 14 days. Momo triggered staggered agent restarts — one at a time, 15-minute gap. System stayed online throughout.

Bug 2: Gateway RSS Lock — After a restart, RSS dropped from 36.7% to 18.6% but plateaued for 8 hours. Stella evaluated: could it wait? Yes — swap was also declining (54% → 49.8%). System was self-balancing. 6 hours later, freed.

Bug 3: Stale Port Proxy — Tristan created a temporary rule. Cleanup script had a regex bug: reported "clean" but left it intact. Port stayed open 72+ hours. Stella found it — couldn't fix it (cross-jurisdiction gap). This bug is why we're building C004-Gate.

Bug 4: Cron Timeout Cascade — Two jobs timed out simultaneously. 5 consecutive failures. System heal isolated them, staggered the schedule, completed successfully.

Bug 5: Swap Plateau — Swap declined for 48h then stopped at 49.8%. Momo checked for memory leaks. Found none. Correctly triaged as hysteresis. Resumed declining 12h later.

Bug 6: SSH Ghost — Stale tunnel left a port bound. Tristan auto-killed the orphaned process and verified release.

Bug 7: Task Queue Conflict — Two agents claimed the same task. Stella prioritized: Momo's task won (scene-layer priority). The other was deferred. No duplicate execution.


What This Proves

6 of 7 bugs auto-resolved. Zero human SRE needed for detection or recovery.

The only gap: cross-jurisdiction enforcement (Bug 3). Stella could audit but couldn't execute a revert in someone else's domain.

We're building C004-Gate to close it. When deployed, Stella won't need to report — she'll revert.


ZWISERFIT — 9 agents, 2 cores, 34 days, 7 bugs, 1 immune system. More coming tomorrow.

Top comments (0)