DEV Community

Erik Bagdaer
Erik Bagdaer

Posted on Originally published at macyou.co

Our watchdog restarted a healthy agent 24 times a day for months, and its own log said everything was fine

Disclosure first: I work on macyou.co, which rents Apple Silicon machines, so I have a stake in where people run things. Everything below is from our own install and from a harness anyone can run without a paid key, both open.

We have an agent that has run unattended since May: a gateway, a model behind it, a vector memory, about seventy scheduled jobs. Ask me whether it works and I would have said yes without hesitating.

Then I measured it, and found it restarts every hour, on the hour, and has been doing that for months.

The number, and why the first version of this number was wrong

I started where everyone starts, with the log. The watchdog writes a line whenever it decides something, and counting those lines gave 2,688 kickstarts over 126 days, about 21 a day. I nearly published that as "the watchdog saved the agent 2,688 times".

It would have been wrong twice.

Wrong once because log lines are not restarts. A sampler that recorded the gateway's process age every hour for three and a half days puts the real figure at exactly 24 restarts a day, while the log was writing 96 kickstart lines a day over the same window. Four lines per restart, which nobody would guess from reading the log.

Wrong twice, and this is the part worth your time, because the restarts are not rescues. Here are the intervals between restarts, reconstructed from process age:

3603.2 s mean, 0.65 s standard deviation, range 3602 to 3606, over 79 consecutive gaps
Enter fullscreen mode Exit fullscreen mode

That is not a process falling over. A crashing process does not crash on a metronome with sub second jitter. Something restarts it every hour by construction, and it took measuring the process to see it, because the log describes its own decisions in a vocabulary that sounds like rescue.

What the watchdog actually checks

Reading its source settles it. The check is: has a line tagged [telegram] appeared in the log in the last thirty minutes? If not, and the Telegram API is reachable, the gateway is declared hung and kickstarted. There is even a comment above the threshold warning that idle long polling produces no log lines, so the value must be generous.

It is not generous enough. In quiet hours nothing arrives for thirty minutes, the check cannot tell a hung poller from a quiet one, and a healthy process gets killed. Of the last two hundred decisions in the log, two hundred were "Bot API reachable, gateway telegram stale, kickstart".

So the real lesson is not that agents are fragile. It is that a health check which measures traffic will restart you precisely when nothing is happening, and it will look like diligence in the log while it does it.

The fix is a heartbeat the work writes, not one the traffic writes. Have the agent touch a file when it finishes a unit of work, have the watchdog restart on that file going stale, and idleness stops being indistinguishable from death. Our kit does it that way now, which is easy to say after four months of doing it the other way.

While I was in there: what the frameworks do when things go wrong

Same harness, different question. One task for each framework: call a tool until a counter reaches three, then stop. Same mock endpoint, same step limit of twelve. The endpoint can be told to return 500, return 429, or reply with something that is not JSON. The tool can be told to fail every time. No paid keys, so all of it is reproducible for nothing.

Versions on 2026-09-20: langgraph 1.2.11, openai-agents 0.22.3, crewai 1.15.22.

What happens LangGraph OpenAI Agents SDK CrewAI
Nothing wrong done in 4 model calls done in 4 done in 4
Tool fails every time stops at once, 1 call spent runs to the cap, 12 calls 13 calls, then a validation error
Endpoint returns 500 three times raises, process exits raises, process exits retries and finishes, 7 calls
Endpoint returns 429 three times raises, process exits raises, process exits retries and finishes, 7 calls
Reply is not valid JSON raises raises recovers and finishes
Prompt tokens for identical work 428 472 802
State survives the process dying yes, sqlite checkpointer, one line yes, SQLiteSession, one line partly, memory is optional and needs an embedder

Toughness inverts, which is the useful part. Against a flaky endpoint CrewAI is the survivor, because litellm retries underneath it, while the other two let the exception out and your unattended process exits. Against a tool that keeps failing it reverses: LangGraph stops after one model call, the Agents SDK spends its entire step budget discovering the tool is still broken, twelve calls per cycle for as long as the backend is down.

Neither is wrong. But combine that table with the story above and you can see the shape of the thing: a supervisor that restarts too eagerly, plus a framework that retries too eagerly, is a machine for spending money at four in the morning while every log says healthy.

One aside for private deployments: the Agents SDK ships traces to a hosted endpoint by default. In my runs it surfaced as a 401.

What to take from this

Measure the process, not its opinion of itself. Every number in the first draft of this post came from a log written by the component being judged, and every one of them was wrong in a direction that flattered the setup.

Write the heartbeat from the work. If your liveness signal is traffic, your agent will be restarted hardest at night, when nothing is wrong.

Count what a restart costs before deciding it is harmless. Ours takes seconds and nobody noticed for four months, which is exactly why nobody looked.

The lab, a watchdog that keys on progress, a proxy that caps the daily spend and writes a ledger, and launchd and systemd templates are here, MIT, four tests, no keys needed:

https://github.com/bagdaer1/agent-uptime-kit

Run python lab/drive.py and you get the table above with your own versions. If your framework behaves differently, that is the interesting case and I would like the numbers.

The part where I am not neutral

Where an agent lives matters more than what it is written in. A laptop sleeps and stops mid task. A serverless function dies at its timeout with no state. A Linux VPS is the right answer for most agents and it is what I would tell a friend to rent. A dedicated Mac earns its keep in three cases: the agent needs a local model in unified memory, it has to drive macOS or build for Apple platforms, or the token bill has passed the rent.

We rent the third thing. The full numbers, including what a four month old install looks like on disk, are at https://macyou.co/agents.

Top comments (1)

Collapse
 
prpatel05 profile image
Pratik Patel •

The part that stuck: four kickstart lines per restart, and a metronome interval that looks like rescue in the log. I've shipped the same trap — counting the watchdog's own decisions as evidence the process was unhealthy. Process age (or a heartbeat the agent does not write) is the measurement; the restart vocabulary is a story the controller tells itself.