A green status from a scheduled agent run tells you exactly one thing: the process started and exited without an infrastructure error. That's it. It does not tell you the task got done.
An agent that can't find its input, misreads the repo it cloned, or just decides there's nothing to do this time will exit cleanly. To your scheduler, that looks identical to a run that actually did the work. We ran a three-brand automation fleet where three separate scheduled jobs reported green for days — one week, in the worst case — while writing nothing at all. Nothing paged us. Nothing looked wrong from the outside.
Make the artifact the definition of success, not the exit code
The fix is smaller than it sounds: every scheduled job has to name, up front, the file it's supposed to write. A run is only ok if it wrote that file. Anything else — the job runs clean but produces nothing — gets recorded as no_op, which is a distinct status from ok, not a footnote inside it.
If you want to check this without touching the job's internals, hash the target file before the run and after. Unchanged hash, non-zero exit for your monitor. It's crude, but it turns a silent no-op into something your alerting can actually see.
The timestamp bug that made this worse
Two of our jobs were logging their scheduled cron time as the run time instead of the time they actually executed. Staleness gets computed from that field, so a job that silently died would still show a timestamp from a few hours ago — reading as fresh, healthy, on schedule. Always record wall-clock time captured at the moment of execution, never the time the schedule intended.
One monitor isn't enough by itself
A supervisor that only checks "did the jobs that ran succeed" misses the job that stopped running entirely. What actually catches that is reconciliation in both directions: read the declaration of every job that's supposed to exist, then diff it against the run ledger. A declared job with no recent entry is stale. An entry in the ledger that nothing declares is invisible work nobody's watching — and invisible work is exactly the kind that dies without anyone noticing.
Watch your false-alarm rate, or the real alert gets ignored too
We shipped a supervisor, and on one day it fired five alerts. Four were wrong — it had counted a single shared worker as three separate per-brand jobs and flagged the "missing" copies. The next day, a real outage in the same layer produced an alert that looked exactly like the noise from the day before. We nearly dismissed it. If your monitoring cries wolf, the fifth alert doesn't get read any more carefully than the four that came before it — mark findings by confidence and only surface the ones you can actually prove, even if that means holding some back.
The check that actually changes something is the one that blocks something
We ran a weekly accuracy check for two months. It produced reports every week. Two of those weeks led to a fix. The other weeks the same findings just... reappeared, because nothing was obliged to act on them — reading a report and doing something about it were two separate, optional steps. What changed that: making the check gate a deploy once a finding passed its grace period. An alarm nobody's required to respond to is a suggestion. An alarm that blocks the pipeline is the only kind that reliably gets fixed.
The short version
Green means "didn't crash," not "did the thing." If you're running scheduled agents against anything you actually care about, you need: an artifact-based success check, real execution timestamps, two-directional reconciliation between what's declared and what ran, confidence-scored alerts, and at least one check that's wired to block something rather than just report it. We had to learn all five the hard way, in that order, over about two months of running this in production. Distribution Autopilot Kit is the packaged version of exactly this setup — the artifact contract, the supervisor, the gated check — extracted from the same fleet these incidents happened on. But the five rules above cost nothing and apply no matter what's running your schedule.
Full answer: https://agentkitworks.com/answers/why-do-scheduled-ai-agents-report-success-but-do-nothing
Top comments (0)