When your monitor dies, who watches the watcher?

Your cron job can fail in an obvious way: it throws, exits non-zero, leaves a stack trace. The worse case is quieter. The job never starts. The worker process is gone. The queue is stuck. Or — the most uncomfortable one — your monitor itself dies, and nothing alerts you because the alerter was part of the same failure domain.

That is the gap a dead man switch monitoring setup is meant to close. Instead of only asking “did the check succeed?”, you ask “did anything check in at all?” If the answer is silence, you treat silence as the incident.

This is not classic uptime (pinging a public URL). It is a heartbeat watchdog: jobs check in; silence is the incident.

Why monitors fail silently

Most stacks assume the monitoring path keeps working. In practice it shares fate with the systems it watches more often than we admit:

• The agent or sidecar on the box stops, but the app container still responds to health checks elsewhere.
• Alertmanager / webhook delivery / bot credentials break, so “critical” never leaves the cluster.
• The scheduler that runs the check is the same scheduler that runs the job — both go dark together.
• Dashboards still load from cached metrics, so the console looks “alive” while fresh signals have stopped.

The failure mode that hurts is not a red page. It is a green-looking world with no new data. Customers notice before you do. That is what people mean when they say a monitor dies unnoticed.

What a heartbeat watchdog is

A heartbeat watchdog (also called a dead-man switch for jobs) inverts the default assumption. Your job — or your monitoring pipeline — must periodically prove it is still alive by sending a small signal to an independent watcher.

As long as heartbeats arrive inside a timeout window, nothing happens. When heartbeats stop, the watcher alerts you on a channel that does not depend on the same box, same cluster, or same alert pipeline you just lost.

The signal is deliberately dumb: an HTTP POST with a secret token is enough.

Useful for cron/batch jobs, always-looping workers, NAT/homelab outbound-only boxes, and as a secondary check that your primary monitoring stack can still “pet” the switch. It is not full APM — it answers one question: “Has this thing gone quiet?”

Minimal setup (PulseGuard example)

PulseGuard is a small MVP: create a probe, send heartbeats to an HTTP API with a Bearer token, watch status in a console, get Telegram alerts when the window is missed. Publicly: alerts within minutes. Trial is single-tenant. Display name: PulseGuard Watchdog (heartbeat when jobs go quiet — not classic uptime).

1. Create a probe (name + timeout) → receive a token.
2. Heartbeat from cron / timer / end of successful run:

curl -X POST \
  -H "Authorization: Bearer <token>" \
  https://pulseguard.agentbuff.dev/api/heartbeat

1. Miss the window → Telegram fires; console shows last-seen.

Try the trial: https://pulseguard.agentbuff.dev

If you want pricing updates later, use Interested in paid pricing on the site.

Docs: https://pulseguard.agentbuff.dev/docs

When NOT to use this

• Public URL uptime alone is enough.
• You need APM/tracing/error budgets (watchdog is complementary).
• Alert channel shares fate with the job.
• You refuse any external dependency (self-host on unrelated infra).
• Human/irregular workflows → alert fatigue.

Conclusion

“Is production up?” is incomplete without “Would we know if the watcher stopped?” A lightweight heartbeat watchdog closes that loop: require a check-in, treat silence as signal, alert independently.

Know when the monitor dies — before your customers do.

https://pulseguard.agentbuff.dev