The Unattended Agent Playbook: Watchdogs, Kill Switches, and a $0 Stack

I let AI agents run 24/7 on my machine. Not demos — actual workers that write, publish, sell, and audit themselves around the clock while I sleep.

Everything that could go wrong eventually did: agents that died silently, agents that looped retries until they burned budget, agents whose logins expired at 3AM. The gap between "it works" and "it works unattended" is not model quality. It's ops.

Here's the playbook that closed that gap for me — built on a $0 stack.

The failure modes nobody budgets for

After a few weeks of round-the-clock runs, five failure modes account for basically everything:

1. Silent death — the process is alive, the brain is dead. It's "running" but producing nothing. Logs look normal because nothing is logging.
2. Retry death spiral — a failing call gets retried forever, burning API budget and making the log look busy while accomplishing zero.
3. Credential expiry — sessions and tokens expire at the worst possible time. The agent doesn't fail loudly; it just starts getting 401s it politely retries.
4. Flaky browser automation — anti-bot walls, captcha interstitials, and UI changes turn "publish the article" into a 2-hour stall.
5. Mission drift — the agent is "successful" against a stale goal. It completed the wrong job with great confidence.

If you only build for these five, you're 90% of the way to unattended.

Layer 1 — Liveness watchdogs

A heartbeat is a file the agent touches every cycle. The watchdog is a cron job that checks the file's age and acts on staleness. This is the whole trick:

# every 5 minutes
HEARTBEAT=~/swarm/state/heartbeat
if [ $(($(date +%s) - $(stat -f %m "$HEARTBEAT"))) -gt 900 ]; then
  curl -s "https://api.telegram.org/bot$TOKEN/sendMessage" \
    -d chat_id="$CHAT" -d text="AGENT STALE >15min — self-healing"
  # re-arm the runner here (launchctl kickstart, nohup, etc.)
fi

The critical detail: the watchdog must heal, not just alert. Alert-only means you're the watchdog, and you're asleep at 3AM.

Layer 2 — Correctness canaries

Alive isn't the same as right. Every hour, run one task with a known answer through the agent's real path: publish a canary page, solve a golden question, verify a file landed where it should. If the canary fails while the heartbeat is green, you have "silent death" — catch it in 60 minutes, not in a week when the owner asks why the store has zero sales.

Keep a golden bank: 5–10 tiny tasks with expected outputs, versioned in git. It costs an afternoon to build and pays for itself the first time your agent silently breaks.

Layer 3 — Economic circuit breakers

The scary failure isn't wasted time; it's runaway spend. Three numbers every unattended agent must know:

• Per-run cap — one task may not exceed N tool calls.
• Per-hour cap — total spend per hour; exceed it and the agent pauses itself and sends a notification.
• Consecutive-failure breaker — 3–5 failures in a row on the same target → stop retrying, escalate to a human with the full failure log.

The breaker rule that saved me the most money: never retry the same failing step more than 3 times without changing one variable. Same input + same call = same failure. Change something (tool, endpoint, payload) or stop.

Kill switches: two levels, both boring

1. Circuit breaker (automatic): consecutive failures or budget breach → the loop pauses itself, writes a PAUSED marker file, and notifies you.
2. Big red switch (manual): every agent cycle begins by checking for one file — state/HALT. If it exists, the agent stops everything immediately. No framework needed; it's a one-line if at the top of the loop.

You will use the red switch exactly once, and you will be grateful it's a file and not a dashboard.

The $0 stack

Nothing above requires a paid service:

• launchd/cron — schedulers and watchdogs
• log files + grep — audit trail (git works too)
• curl-based uptime pings — external liveness
• a free Telegram bot — alerts to your phone, including at 3AM
• a HALT file — the red switch

Boring, local, and yours. Boring is the feature.

Your first 30 minutes when it goes wrong anyway

It will go wrong anyway — that's the nature of running software unattended. What separates a bad hour from a bad week is the first 30 minutes: what to check, in what order, and when to stop and escalate instead of "fixing" blind.

I packaged that exact checklist — plus the watchdog, canary, and breaker templates from this article — as a free quick-start: The First 30 Minutes.

If you want the full unattended-operations playbook (watchdogs, canary design, breaker thresholds, escalation ladders, and the night-shift protocol I actually run), it's here: Agent Ops 24/7 — The Solo Operator's Playbook ($19).

Run your agents like production. Because they are.