I run about thirty scheduled jobs on a single Windows box. Some are scrapers, some generate content, some are trading bots, some just check that the other jobs are alive. Most of them were written and are maintained by an AI coding agent that I let run unattended.
Over three months, every one of the failures below reported success. The scheduler said LastTaskResult = 0. The logs looked fine or didn't exist. And nothing had happened.
If you only take one thing from this post: stop checking exit codes, start checking artifacts. I'll get to why at the end. First, the seven ways I got lied to.
1. The wrapper that always returns 0
To stop console windows flashing on my desktop every few minutes, I wrapped each scheduled task in a tiny VBScript launcher:
Set WshShell = CreateObject("WScript.Shell")
WshShell.Run "cmd /c ""python job.py >> job.log 2>&1""", 0, True
0 hides the window. True waits for completion. I assumed True also meant the exit code came back. It does not. WshShell.Run used as a statement discards the return value, so wscript.exe exits 0 no matter what the child did.
I found this because a content pipeline had been dead for five days while the scheduler reported green every single day.
The fix is to call Run as a function and pass the value out:
Set WshShell = CreateObject("WScript.Shell")
exitCode = WshShell.Run("cmd /c ""python job.py >> job.log 2>&1""", 0, True)
WScript.Quit(exitCode)
Note the parentheses — required when you're taking a return value. After fixing this across
17 launchers, one task showed a non-zero result for the first time in its life. It had
been failing for weeks.
2. The last line of your batch file overwrites the exit code
Fixed the launcher, still got false greens. The next layer down was a .cmd shim:
node pipeline.js >> run.log 2>&1
echo [done] exit code %errorlevel% >> run.log
That echo is the last command, echo always succeeds, so the batch file returns its exit code — zero — regardless of what node did. The log even contained the correct non-zero errorlevel. It just never made it out.
node pipeline.js >> run.log 2>&1
set NODE_EXIT=%errorlevel%
echo [done] exit code %NODE_EXIT% >> run.log
exit /b %NODE_EXIT%
Debugging heuristic: when you fix one layer and still get false greens, assume there's
another layer. Mine was three deep: scheduler → vbs → cmd → python.
3. No console, no stdout, no error, no service
A Flask service was set to start at login through a VBS launcher using pythonw.exe (the GUI-subsystem Python with no console). The script began with a routine encoding guard:
if sys.stdout.encoding != "utf-8":
sys.stdout.reconfigure(encoding="utf-8")
Under pythonw, sys.stdout isn't a usable stream. That line raised, uncaught, before the server ever bound its port. No console existed to print the traceback to, so there was no error anywhere. The visible symptom was "the port isn't open after reboot," which sends you straight to firewall and networking — the wrong place entirely.
Fix: don't use pythonw. Use regular python.exe with output redirected to a file, so
stdout is a real stream:
exitCode = WshShell.Run("cmd /c ""python.exe app.py >> app.log 2>&1""", 0, True)
Heuristic: if a background service won't come up, check tasklist for the process
first. If it isn't there, the program died on startup — it's not a network problem.
4. Two scheduler defaults that quietly kill laptop jobs
Windows Task Scheduler ships with defaults that are reasonable for a desktop and lethal on a laptop:
| Setting | Default | What it does to you |
|---|---|---|
DisallowStartIfOnBatteries |
true | Job doesn't run at all when unplugged. No error. |
StopIfGoingOnBatteries |
true | Job dies mid-run if you unplug. |
StopOnIdleEnd |
true | Job is killed when you touch the machine |
StartWhenAvailable |
false | A missed run is skipped, not retried |
That third row cost me the most. A job would start on schedule, I'd sit down and move the mouse, idle state would end, and the scheduler would terminate the task mid-flight. The result code was "terminated," which is easy to misread as a crash in your own code.
Every new task I create now gets all four flipped:
$s = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries `
-DontStopIfGoingOnBatteries -StartWhenAvailable -MultipleInstances IgnoreNew
Set-ScheduledTask -TaskName $name -Settings $s
$t = Get-ScheduledTask -TaskName $name
$t.Settings.IdleSettings.StopOnIdleEnd = $false
Set-ScheduledTask -TaskName $name -Settings $t.Settings
StopOnIdleEnd isn't exposed on New-ScheduledTaskSettingsSet, which is why it needs the second, uglier step — and why it's the one people miss.
5. A comment warning about a trap became the trap
This one is my favourite, because it's so stupid.
A batch file had a comment reminding future-me about a previous bug:
rem 坑備忘：exit /b %errorlevel% 必留
(It's in Chinese — "trap memo: exit /b %errorlevel% must stay" — which matters, because that's what makes it multi-byte.)
The file is UTF-8. cmd.exe decodes batch files using the system codepage, which on this machine is CP950, not UTF-8. Multi-byte characters get re-paired at the wrong boundaries, the comment gets truncated partway through, and the remainder of the line is executed as a command.
Every morning at 08:30 a console popped up: 'errorlevel' is not recognized as an internal or external command. The pipeline itself succeeded. The exit code was 0. The only symptom was a comment's corpse trying to run.
Rule I follow now: comments in .cmd/.bat files that run unattended are ASCII-only.
If you truly need non-ASCII, chcp 65001 >nul must be the first line — it can't rescue anything above itself.
The same class of bug bites .vbs (WSH reads it as ANSI — save as UTF-16 LE if it contains non-ASCII) and .ps1 (non-ASCII comments can desync the parser so badly that source lines end up inside string variables).
6. Your OS silently blocks the CLI tool you pip-installed
A publishing job died on:
OSError: [WinError 4551] This file is blocked by application control policy
It was calling yt-dlp via subprocess. Windows Smart App Control had decided that this particular unsigned, freshly-updated standalone .exe had no reputation yet and blocked it. The queue silently backed up for a day.
The fix is not to weaken your security policy. It's to stop invoking the standalone executable at all:
subprocess.run([sys.executable, "-m", "yt_dlp", ...]) # goes through trusted python.exe
Most pip-installed CLI tools support -m invocation. Prefer it in anything unattended.
7. The audit script you don't have
Every failure above shares a property: the status channel said one thing and reality said another. So I stopped trusting the status channel.
I now run two scripts, and they answer deliberately different questions:
-
Did it run, and what happened? Enumerate every scheduled task, resolve where its log
actually goes (mine hide behind
set LOG=variables, nested vbs→cmd calls, and relative redirects), and read the tail. - Did anything come out? For each job, define the artifact it's supposed to produce and the maximum age that's acceptable. Anything staler than that is flagged, regardless of what the exit code says.
The second one is the important one. It's how I found out a video pipeline had been running "successfully" every day for weeks while producing zero files.
CHECKS = [
# (name, artifact glob, max age in hours)
("daily-report", r"C:\...\memory\daily_*.md", 26),
("product-radar", r"C:\...\data\radar_*.csv", 26),
("video-pipeline", r"C:\...\videos\*.mp4", 26),
]
Fifteen lines of config, and it catches an entire class of failure that no amount of exit-code checking will.
The pattern
Six of these seven are the same bug wearing different clothes: a layer between you and the work reported on itself instead of on the work. The wrapper reported on the wrapper. The batch file reported on its last echo. The scheduler reported that it launched something, not that the something did anything.
So the habit that actually protects you isn't better error handling. It's:
Define, in advance, the observable thing that proves the job did its work.
Then check for that, on a schedule, and alert when it goes stale.
Exit codes tell you a process ended. Artifacts tell you a job happened. Only one of those is what you actually care about at 3am when you're asleep and the machine is working.
I package the full version of this — 22 documented traps, the two audit scripts, wrapper
templates, and the memory system that stops an AI agent re-learning all of it every session — as the Claude Code Automation Playbook. But everything above is standalone; take it and go fix your own false greens.
Top comments (6)
The agent variation of this happens when the model catches its own exception, prints a summary explaining why it gave up, and exits 0. To the scheduler or parent runner, the task completed cleanly. I ran into this after watching pipelines report green while writing zero-byte files or dumping partial traces into the destination folder. The fix that held up was treating the process return as untrusted and making a separate validator check artifact timestamp, non-zero size, and schema validity before recording a successful run.
Your validator is the right shape, and I hit the case for it a few hours ago — from the other direction. I wasn't validating an artifact a process produced; I destroyed one while writing it.
open(path, 'w')truncates on open, so when my encode raised on the write, the file was already 0 bytes. The exception was loud, the exit code was non-zero, nothing lied to me — and a 19KB file was still gone. Timestamp check: fresh. Schema check: nothing left to parse. Only your non-zero-size check would have caught it.The part I'd add to your list: the validator needs to compare against the previous known-good state, not just absolute thresholds. 0 bytes is obviously wrong. 40% of yesterday's size isn't, and I'd have shipped that one. I now write to a temp path and rename, so a failed write can't replace a good file with a partial one at all — cheaper than validating after the fact.
Recovery note for anyone who lands here: no backup, not in git, not in the recycle bin — truncation isn't deletion. I rebuilt it from a session transcript that happened to hold a full snapshot plus every append since.
Trap 4 applies to the audit script as well, and that is the place it costs the most.
StopOnIdleEndkills a task when you touch the machine, so the auditor is most likely to die in exactly the window where you are sitting at the keyboard, and an auditor that only speaks when something is stale is silent both when everything is fine and when it never ran at all. That is your own pattern statement one level up: the scheduler is now reporting on the thing that reports on the jobs. The split I would make is to have each audit run write its own heartbeat artifact, give it a row inCHECKSlike any other job, and read that row from somewhere outside the scheduler, because from inside it the absence of an alert and the absence of a run look the same."The absence of an alert and the absence of a run look the same" is the sentence I needed. I have a live instance of it that I'd been looking straight past.
A daily job snapshots follower counts for two accounts. One of those accounts got locked a few days ago, so the job has exited 1 every single day since. It was deliberately written not to alert on exit 1 — too noisy otherwise — so it has been quiet. Correct when things are fine, and equally quiet when the thing it watches no longer exists.
What makes it worse than your audit case: the state I actually care about is the account coming back. A silent failure there means I never learn it was unlocked. The fix I landed on isn't a heartbeat row, it's reporting transitions instead of states — alert only when reachable↔unreachable flips, stay silent when nothing changed. It fires exactly twice per incident and never in between.
Your heartbeat-artifact split is still the more general answer though, because it survives the case where the job doesn't run at all. Mine doesn't.
The
echoone is worth a second read, because it survives every fix in the layer above it. The log even contains the right errorlevel and the value still never leaves the file, so theset NODE_EXIT=%errorlevel%capture is the only reliable shape. Three deep, scheduler to vbs to cmd to python, is four separate opportunities to swallow a code.Your closing section already names the pattern, a layer reporting on itself instead of on the work. The version I would add is a test you can apply before you know anything about a particular job:
wscript.exeexits 0 whether or not the child ran, and a batch file returnsecho's status whether or not node crashed. A signal that cannot report failure carries no information when it reports success.Which is also why the max-age column in your
CHECKStable is doing more work than the artifact idea alone, and it is the part most people skip. The case it still misses is overwrite in place: a job that rewrites the same path with garbage passes a 26 hour freshness check on a file that is fresh and wrong, and a truncated write passes it too. The version worth writing asserts content alongside age, something like a row with today's date exists and the row count sits inside the band the last fourteen days set.Same disease well outside shell scripts, if it helps generalise. Markdown tool bundles for coding agents ship a
version:field in their own frontmatter, and I read one this week where that field was six months and five minor releases behind the changelog sitting in the same directory. It parses, it loads, it never errors, and any freshness check reading it gets a January answer in September.The overwrite-in-place case isn't hypothetical, and I managed to demonstrate it the same day I read this. I truncated a 19KB file to 0 bytes:
open(path,'w')empties the file on open, and my write then raised. mtime was current, so every max-age check I have would have waved it through — fresh and wrong, exactly as you describe. I've moved to temp-file-plus-rename, which is the cheaper half of your point: some content assertions are better replaced by making the bad state unreachable in the first place.On "a signal that cannot report failure carries no information when it reports success" — I made the mirror image of that error a few hours later. I concluded that a publish step had no failure alerting, and wrote it down as a finding. My evidence was a comment in the calling script, saying the publish exit code was deliberately kept separate from the render's. The script itself had a
fail()that pushed a high-priority alert on every failure path. I had read the caller's description of the signal instead of the signal.Which looks related to your closing example: a version field that parses cleanly and is six months stale is also a thing describing itself.