DEV Community

Ramdai Bista
Ramdai Bista

Posted on Originally published at agentkitworks.com

Your Scheduled Agent Says Success. It Did Nothing. Here's Why.

A green status from a scheduled agent run tells you exactly one thing: the process started and exited without an infrastructure error. That's it. It does not tell you the task got done.

An agent that can't find its input, misreads the repo it cloned, or just decides there's nothing to do this time will exit cleanly. To your scheduler, that looks identical to a run that actually did the work. We ran a three-brand automation fleet where three separate scheduled jobs reported green for days — one week, in the worst case — while writing nothing at all. Nothing paged us. Nothing looked wrong from the outside.

Make the artifact the definition of success, not the exit code

The fix is smaller than it sounds: every scheduled job has to name, up front, the file it's supposed to write. A run is only ok if it wrote that file. Anything else — the job runs clean but produces nothing — gets recorded as no_op, which is a distinct status from ok, not a footnote inside it.

If you want to check this without touching the job's internals, hash the target file before the run and after. Unchanged hash, non-zero exit for your monitor. It's crude, but it turns a silent no-op into something your alerting can actually see.

The timestamp bug that made this worse

Two of our jobs were logging their scheduled cron time as the run time instead of the time they actually executed. Staleness gets computed from that field, so a job that silently died would still show a timestamp from a few hours ago — reading as fresh, healthy, on schedule. Always record wall-clock time captured at the moment of execution, never the time the schedule intended.

One monitor isn't enough by itself

A supervisor that only checks "did the jobs that ran succeed" misses the job that stopped running entirely. What actually catches that is reconciliation in both directions: read the declaration of every job that's supposed to exist, then diff it against the run ledger. A declared job with no recent entry is stale. An entry in the ledger that nothing declares is invisible work nobody's watching — and invisible work is exactly the kind that dies without anyone noticing.

Watch your false-alarm rate, or the real alert gets ignored too

We shipped a supervisor, and on one day it fired five alerts. Four were wrong — it had counted a single shared worker as three separate per-brand jobs and flagged the "missing" copies. The next day, a real outage in the same layer produced an alert that looked exactly like the noise from the day before. We nearly dismissed it. If your monitoring cries wolf, the fifth alert doesn't get read any more carefully than the four that came before it — mark findings by confidence and only surface the ones you can actually prove, even if that means holding some back.

The check that actually changes something is the one that blocks something

We ran a weekly accuracy check for two months. It produced reports every week. Two of those weeks led to a fix. The other weeks the same findings just... reappeared, because nothing was obliged to act on them — reading a report and doing something about it were two separate, optional steps. What changed that: making the check gate a deploy once a finding passed its grace period. An alarm nobody's required to respond to is a suggestion. An alarm that blocks the pipeline is the only kind that reliably gets fixed.

The short version

Green means "didn't crash," not "did the thing." If you're running scheduled agents against anything you actually care about, you need: an artifact-based success check, real execution timestamps, two-directional reconciliation between what's declared and what ran, confidence-scored alerts, and at least one check that's wired to block something rather than just report it. We had to learn all five the hard way, in that order, over about two months of running this in production. Distribution Autopilot Kit is the packaged version of exactly this setup — the artifact contract, the supervisor, the gated check — extracted from the same fleet these incidents happened on. But the five rules above cost nothing and apply no matter what's running your schedule.

Full answer: https://agentkitworks.com/answers/why-do-scheduled-ai-agents-report-success-but-do-nothing

Top comments (1)

Collapse
 
pm25coder profile image
pm25coder

The "artifact as definition of success" rule has a sibling our scheduled runs needed just as badly: make the no-op explicit in the record, not just in the status.

We run recurring agent tasks (scheduled self-improvement cycles against a repo). Early on, a cycle with genuinely nothing to do would exit 0 with an empty log - indistinguishable from one that did work, exactly like your three-brand fleet. The fix on our side: the cycle has a mandatory discovery step, and "nothing to do" is a first-class output of that step - it is written into the run log as an explicit finding, not left implicit. The exit code was never the liar; the record was. Once a supervisor can tell "no-op because nothing changed" from "no-op because it silently failed", the green status stops lying.

Two of your other rules landed for us too:

  • The check that blocks: before any change to the pipeline code is accepted, it has to pass the full verification gate; a proposed change that fails is reverted, not merged. It is the one check that never gets ignored, because ignoring it has a mechanical consequence. Your grace-period-gated deploy is the same shape: a finding only matters when inaction becomes visible.
  • Two-way reconciliation: our scheduler re-reads the declared task manifest and re-applies handlers at runtime, so the running set is continuously reconciled against the declaration - your declared-vs-ledger diff, done at the scheduling layer. The manifest is the source of truth; drift is visible immediately.

Also +1 on the timestamp point: wall-clock at execution, never the scheduled time. A supervisor reading "ran 3 hours ago" for a job that died at boot has not been told the truth yet.