A tutorial on detecting a python bot that's deadlocked or stuck without crashing, using heartbeat monitoring — with a working code example.
How to Detect a Frozen Python Bot (Not Just a Crashed One)
Most reliability setups only catch one kind of failure: the process exiting. A bot that's deadlocked, stuck on an API call with no timeout, or spinning in a broken loop never exits at all — it just sits there, "running" by every normal check, doing nothing useful. This tutorial covers detecting that specific failure mode, which standard crash recovery is structurally blind to.
Table of Contents
- Why Crash Detection Misses This
- What a Frozen Bot Actually Looks Like From Outside
- Step 1: Install StayPresent
- Step 2: Add a Heartbeat Call to Your Bot's Loop
- Step 3: Set a Heartbeat Timeout
- Step 4: Choosing the Right Timeout Value
- Step 5: Confirming It Works
- What Happens When a Hang Is Detected
- FAQs
- Conclusion
Why Crash Detection Misses This
Standard restart logic reacts to a non-zero exit code. A deadlock or an infinite loop with a broken exit condition never produces one — the process is technically alive the entire time it's stuck, so nothing about exit-code-based monitoring ever triggers.
What a Frozen Bot Actually Looks Like From Outside
From a hosting dashboard's perspective, a frozen bot and a healthy one look identical: process running, memory allocated, no errors in the log because nothing is throwing one. The only real signal something's wrong is the complete absence of anything happening — no new messages processed, no API calls made, no progress at all.
Step 1: Install StayPresent
pip install staypresent[prod]
Step 2: Add a Heartbeat Call to Your Bot's Loop
In your bot script, call staypresent.heartbeat() once per iteration of your main loop:
# worker.py
import time
import staypresent
while True:
staypresent.heartbeat()
do_one_unit_of_work()
time.sleep(1)
Place it near the top of each iteration — this way, if the work itself hangs, the next expected heartbeat is what catches it.
For an async, event-driven bot (discord.py, Pyrogram), call it from a periodic background task instead:
async def heartbeat_task():
while True:
staypresent.heartbeat()
await asyncio.sleep(10)
Step 3: Set a Heartbeat Timeout
In your entry point, pass heartbeat_timeout to run():
# main.py
import staypresent
staypresent.run(
"worker.py",
heartbeat_timeout=30,
)
If worker.py goes more than 30 seconds without calling heartbeat(), StayPresent treats it as hung, terminates it, and restarts it through the normal crash-recovery pipeline.
Step 4: Choosing the Right Timeout Value
Set heartbeat_timeout comfortably above your normal iteration time, including realistic worst-case latency — a slow API call, a large batch. If your loop typically completes in 2–5 seconds, heartbeat_timeout=30 gives generous headroom for occasional slowness while still catching a genuine freeze well before it goes unnoticed.
Step 5: Confirming It Works
You can test this deliberately before relying on it in production — temporarily add a line that sleeps past your configured timeout without calling heartbeat(), and confirm the bot gets terminated and restarted as expected:
# temporary test — remove after confirming
import time
time.sleep(60) # exceeds heartbeat_timeout=30
Watch the "staypresent" logger output; you should see the hang detected and the process restarted.
What Happens When a Hang Is Detected
The behavior mirrors a real crash exactly: the process is terminated, logged, and handed to restart_on_crash/max_restarts/restart_delay — the same recovery path, just triggered by silence instead of a bad exit code. On a recent StayPresent version, this also shows up as its own distinct "unresponsive" incident on the /status page, separate from a plain crash, so you can tell the two failure modes apart after the fact.
FAQs
Does every bot need heartbeat monitoring?
No — it's opt-in. A bot with no heartbeat_timeout set is only monitored for actual crashes, exactly as before.
Can different bots have different timeout values?
Yes — heartbeat_timeout is set per run() call (or per bot, if using bots=[...] with different configurations).
What if my bot legitimately has slow, variable-length operations?
Set the timeout based on your slowest realistic case, or call heartbeat() more granularly between sub-steps of a long operation rather than only once per full iteration.
Conclusion
A frozen process is a genuinely different failure mode from a crash, and it's invisible to anything only watching exit codes. staypresent.heartbeat() combined with heartbeat_timeout closes that gap — your bot proves it's actually making progress, not just technically running.
pip install staypresent[prod]
Top comments (0)