My OpenClaw Agent Was Silently Crashing Every Night. I Wrote 50 Lines of Node to Catch It

My nightly self-improvement cron had been "succeeding" for three weeks. Heartbeat green. Memory file updated. Zero alerts. Then I actually read the session transcript and found out the agent had been silently crashing before producing any output for at least nine days in a row.

The heartbeat system — which is supposed to be the ground truth for "did my agent actually run?" — was lying to me. The cron job fired. The heartbeat-state.json got touched. Nothing actually happened in between.

This is the story of how I caught it, the 50 lines of Node I added to session-review.js to make sure I never miss it again, and the three OpenClaw patterns that produce silent crashes you should be auditing for tonight.

The Symptom: A Cron That Lies

OpenClaw's nightly self-improvement workflow looks like this from the outside:

02:03 cron fires → sub-agent spawns → reads memory/ → writes notes → exits

The subagent-watchdog.timer writes a timestamp to heartbeat-state.json on every fire. If the timestamp is fresh, the system looks healthy.

What I missed: the cron spawns the sub-agent and exits cleanly whether or not the sub-agent did any work. The watchdog measures "did the launcher run?" — not "did the agent succeed?"

So when my agent started crashing with the message:

No user query found in messages. Crashing before processing.

…nothing in the heartbeat infrastructure flagged it. The watchdog saw a green light. Memory/ was empty. The next day's dreaming sweep ran on stale data and produced noise.

Nine days of false-positive health.

The Three Crash Patterns Nobody Warns You About

After digging through ~/.openclaw/agents/main/sessions/, I found three distinct silent-failure shapes in OpenClaw cron-triggered sessions:

1. no_user_query — The cron job's payload.message is empty or malformed. OpenClaw spawns the session, finds nothing to process, and bails before the first turn. Most common cause: a cron add payload with a text field instead of message, or a systemEvent payload whose text was overwritten by a patch.

2. turn failed before producing content — The agent starts a turn, hits an internal error (often a model fallback chain exhausting), and the runtime returns an empty result. The session ends "successfully" from the framework's perspective.

3. tool_budget_exceeded masked by exit-0 — The agent loops on a tool, exhausts its tool-call budget, but the cron wrapper still exits 0 because the spawn itself succeeded.

All three are invisible from the outside. The session file exists, the timestamps look normal, the heartbeat is fresh.

The 50-Line Fix in session-review.js

I extended my nightly session-review script (scripts/session-review.js) with a crash classifier and an action emitter. The core logic:

function classifyFailure(messages) {
  const stats = {
    turns: 0,
    toolCalls: 0,
    crashed: false,
    failureType: null,
  };

  for (const m of messages) {
    if (m.role === 'assistant') stats.turns++;
    if (Array.isArray(m.content)) {
      for (const part of m.content) {
        if (part?.type === 'tool_use') stats.toolCalls++;
      }
    }

    // Detect "turn failed before producing content" — silent crash pattern
    if (typeof m.content === 'string' &&
        m.content.includes('turn failed before producing content')) {
      stats.crashed = true;
    }

    // Flag "No user query found in messages" specifically
    if (typeof m.content === 'string' &&
        m.content.includes('No user query found in messages')) {
      stats.crashed = true;
      stats.failureType = 'no_user_query';
    }
  }
  return stats;
}

Then in the summary printer, I emit an ACTION block whenever a silent crash is detected:

if (r.failureType === 'no_user_query') {
  console.log(`⚠️  SILENT CRASH DETECTED: ${r.sessionKey}`);
  console.log(`  → ACTION: Cron event message lacked a clear user query.`);
  console.log(`    Check the job's payload.message — systemEvent/agentTurn`);
  console.log(`    payloads with no textual user query will crash before`);
  console.log(`    processing. Run: openclaw cron get <jobId>`);
}

The ACTION: line is the part that matters. Without it, "silent crash detected" is just another log line you scroll past. With it, session-review.js becomes a thing that tells you what to do next — which is the whole job of an observability tool.

Wiring It Into the Nightly Loop

The script already runs as part of daily-trend-research.sh at 7 AM. The new behavior:

1. Walk every session file modified in the last 24 hours
2. Classify each one with classifyFailure()
3. If crashed === true, print a SILENT CRASH DETECTED block with the failure type and the remediation hint
4. Exit non-zero if any silent crash was found in a cron-triggered session

The non-zero exit is the trick. A failed session-review now blocks the dreaming sweep from running on top of bad data — so you can't silently accumulate nine days of noise again. Your memory stays honest because the pipeline stays honest.

I added one more belt-and-suspenders check: if stats.toolCalls === 0 && stats.turns === 0 for a cron session, treat it as a candidate silent crash even if neither string pattern matched. Empty sessions are almost never intentional.

What I Learned

Heartbeats measure launcher health, not agent health. A green heartbeat-state.json tells you the cron fired. It does not tell you the agent did work. You need a second signal — transcript inspection — to actually know.

Silent crashes cluster. Once I added detection, I found three crashes I'd missed, all within 48 hours of each other. They were all caused by the same cron patch call that had wiped a payload.message field. One bad patch, nine days of garbage data, zero alerts.

Actionable output matters more than detection. "Silent crash detected" is a log line. "ACTION: check payload.message on job X" is a fix. The difference between the two is whether you actually go look at 7 AM or grab another coffee.

50 lines is enough. I almost wrote a whole observability stack — Prometheus exporter, Grafana dashboard, Slack alerts. Then I remembered: I'm the only consumer. The script prints to stdout. I read it. That's the dashboard.

If you run OpenClaw with any non-trivial cron surface, audit ~/.openclaw/agents/main/sessions/ for the three patterns above. I'll bet you find at least one silent crash you didn't know about. I know I did — three of them.