DEV Community: Youfu Hsu

Giving an AI agent memory is easy. Keeping it true is the hard part.

Youfu Hsu — Wed, 09 Sep 2026 04:38:16 +0000

Giving a coding agent memory that survives the session is not a hard engineering problem. A directory of markdown files outside any repo, one fact per file, an index that gets loaded at the start of every session. That is the whole mechanism, and it takes an afternoon.

I have been running one for about four months across roughly forty projects. The mechanism has never been the problem. In a single working day last week, four separate memory entries turned out to be confidently wrong, and each one had been steering decisions for weeks.

The four

"Schema fix committed locally, not yet pushed." Read as: there is unpushed work sitting on this machine. Actually: that repo has no remote at all and never did. The deploy path was a script hitting the platform's API directly, so nothing was ever supposed to be pushed. The entry had turned a fact about one repo's setup into a phantom to-do that survived several audits.

"Template fatigue in the post generator: fixed and deployed." Green checkmark, verification notes, an A/B comparison in the file. All true. The fix was deployed to a queue worker whose last log line was from six weeks earlier and which had no scheduled task pointing at it. The pipeline that actually posts every morning is a different script that never imports the fixed module. The fix was real, verified, deployed, and had zero effect on anything anyone could see.

"600 platform credits banked, use before November." Actually the credits had already been converted into a different resource with a 30-day expiry, and the deadline was five weeks earlier than the memory claimed. The entry was written on the day of purchase and described the plan, not the outcome.

"Two recordings missed last night." Written by an automated job that judged success from a process exit code. The recordings were on disk, complete, and had already been transcribed. The exit code was a red herring from a downloader that returns non-zero while cleaning up after a perfectly good capture.

Different projects, different weeks, one shape.

What they have in common

None of these were sloppy notes. They were all written carefully, at the moment of the work, by someone (or something) with full context.

Every one of them recorded a judgment and formatted it as a fact.

"Not yet pushed" is an inference from "I did not push." "Fixed and deployed" is an inference from "the deploy script exited zero." "Missed" is an inference from "exit code was 1." The inference was reasonable when it was written. The problem is that once it is a line in a file, the reasoning is gone and only the conclusion is left — and conclusions do not carry their own expiry date.

This gets worse specifically because the memory works. An agent that reads its memory and acts on it is doing the right thing. A wrong entry does not cause an error; it causes confident, efficient work in the wrong direction. My agent spent weeks not investigating the post generator, because the file said it was fixed.

What actually helps

Write the re-check, not just the claim

Any entry that asserts a state — done, fixed, deployed, live, expired — should carry the command that re-establishes it:

✅ Subtitle pipeline deployed and verified 08-18.
Re-check: `grep "card1:" <log> | tail -3` — openers should not
all share one skeleton. If they do, the fix is not on the live path.

Now the claim is falsifiable by anyone who reads it, including a future session with no context. Half of my four would have been caught in seconds by their own re-check line.

Distinguish observed from inferred

I now try to write what was seen, and separately what it was taken to mean:

Observed: deploy script exit 0, target file contains the new function.
Inferred: the fix is live.   ← this is the part that can rot

The inference is usually still right. But when something later contradicts the memory, you know which half to attack first.

Let files outrank memory

When memory and the filesystem disagree, the filesystem wins. This is worth writing as code rather than as a habit. Correcting that "two recordings missed" entry was six lines:

for slug, rec in state.items():
    # source of truth is the artifact, not what the job concluded at the time
    if os.path.exists(f"{WORK}/{slug}/summary.md") and not rec.get("done"):
        state[slug] = {**rec, "done": True,
                       "note": "corrected: exit code misjudged; output is complete"}

Note what it does not do: regenerate the state file from scratch. It patches only the rows contradicted by evidence and leaves everything else alone. I have destroyed hand-maintained data by "regenerating" it from rules before, and that is a genuinely bad afternoon.

Verify from outside the system that made the claim

The strongest habit of the four. A deploy script reporting success is the deploy script's opinion. Ask something that has no stake in the answer.

This cuts both ways, and the false negative is the one that wastes your time. On the same day I fixed the four entries above, I updated 23 listings on a platform, got 23 success responses back, then checked the public search index and saw zero of the changes. Looked exactly like "claimed success, not actually live." It was a caching layer on the search endpoint; the per-item public endpoint had all 23 updates with the correct timestamp.

So: verify externally, but when the external check disagrees, suspect the checker before you rip out the work. The rule is not "the external check is right." It is "one source is never enough."

The part I did not expect

I assumed the risk of agent memory was the model hallucinating something into a file. Four months in, that has not happened once. Everything in my memory directory was written from real work.

The risk is decay. Every entry is a snapshot of a moment, and the world keeps going. A memory system does not fail by filling with lies. It fails by filling with things that were carefully, honestly true in July.

Which means the maintenance job is not "keep the notes tidy." It is: for anything currently steering a decision, go look again.

I write about running unattended automation and AI agents in production, including the parts where the agent — or I — confidently get it wrong. The full system I use for this is the Claude Code Automation Playbook.

Your prompt is feeding the model its own cliché

Youfu Hsu — Tue, 08 Sep 2026 01:08:32 +0000

I have a small pipeline that turns a daily market report into four subtitle cards for a short video. It runs unattended every morning. I checked its output after two weeks and the wording never repeated once.

It still looked machine-generated. Here is what the opening card said on ten consecutive days:

the coin that pumped hardest, but volume is cold?
old coin FIL back on the gainers board?
small caps exploding, majors flat?
up 38% and nobody's buying?
the top gainer is the fakest one?
the coin that rose least is the popularity king
falling price but volume spiking, who's rotating out?
everything red, but this one is up
meme DeFi AI all pumping the same day?
DEX names all agreed to pump together?

Every word is different. Eleven out of fourteen are the same skeleton: a contrast, then a question mark. Read one, it's fine. Read a week, and you can see the machine.

Where it came from

The relevant line of my system prompt:

4. Card 1 is the hook (a cliffhanger or counterintuitive point),
   cards 2-3 carry the concrete numbers, card 4 is a conclusion or a question.

"Cliffhanger." "Counterintuitive." I wrote those words to describe the goal, and the model read them as a description of the form. A counterintuitive point, rendered as a sentence, is "X but Y?" — so that is what came out, every single day.

The model was not being lazy. I put the cliché in the prompt and it did what I asked.

There is a second cause underneath: the model cannot see what it wrote yesterday. Each run is a fresh context. Even if the wording rule were perfect, nothing stops it from reaching for the same structure every morning, because from its point of view every morning is the first one.

Fix 1: stop describing the shape

Replace the adjective with a menu of options, and name the failure mode explicitly:

4. Card 1 is the opener, cards 2-3 carry the concrete numbers, card 4 concludes.
5. Card 1 must open differently every day. Angles to rotate through
   (do not keep picking the same one):
   - name a specific ticker and its number
   - lead with a magnitude (volume, market cap, share of total)
   - lead with a change over time (three days running, week to date)
   - describe the state of a whole group or sector
   - state a fact that is happening, with no evaluation
   Do not write "A is up but B is..." or "...and nobody's buying?" every day.
   A reader who scrolls past three of these should not be able to tell
   they came from the same template.
   Card 1 should not end in a question mark unless the content really is
   an open question.

Two things matter here. Naming the bad pattern in the prompt works better than only describing the good one — the model needs something to steer away from. And listing angles rather than one instruction gives it somewhere to go.

Fix 2: show it what it already said

The structural half needs memory. Read back your own recent output and attach it to the request:

def recent_hooks(n=7):
    """Last n days of card 1, as an avoid-list. Returns '' on any failure."""
    try:
        seen = {}                      # date -> hook, so a same-day rerun overwrites
        for line in open(LOG, encoding="utf-8"):
            m = re.match(r"^\[(\d{4}-\d{2}-\d{2})T[^\]]*\] card1: (.+)$", line)
            if m:
                seen[m.group(1)] = m.group(2).strip()
        hooks = list(seen.values())[-n:]
        if not hooks:
            return ""
        return ("\n\nThese are the openers already used in the last "
                f"{len(hooks)} days. Today's must be clearly different — "
                "not just different words, a different sentence skeleton:\n"
                + "\n".join(f"- {h}" for h in hooks))
    except Exception:
        return ""

Then messages=[{"role": "user", "content": draft + recent_hooks()}].

Three details that are easy to get wrong:

Read from a log you are already writing. My first instinct was a new history.json that the pipeline would append to. That version would have been correct and useless — it starts empty, so the avoid-list does nothing for the first week. The daily log already had card1: ... in it going back a month, so the feature worked on the first run. I had made this exact mistake before on a different pipeline and shipped a de-duplication function that read from an empty directory and therefore returned an empty string forever.

Return empty on failure, never raise. A cosmetic feature must not be able to take down a job that runs while you sleep.

Key by date, not by line. A same-day rerun should replace that day's entry, not add a second one and push a real day off the end of the window.

How to know it worked

Run the old and new prompt against the same input. Same day's report, two runs:

	Card 1
Old	`DEX names all agreed to pump together?`
New	`RAY up 41% in a day`

I added a --dry-cards flag to the production script that stops after the model call — no video render, no upload, no notification — so this comparison runs the code that actually ships rather than a copy of it I pasted into a scratch file. A copy would have proved something about the copy.

That flag introduced one more problem worth mentioning, because it is the kind of thing that bites three weeks later. The dry run writes its cards to the same log that recent_hooks() reads. Test output would have become "history" and been fed back to the model as something to avoid. So the dry mode tags its lines:

log(f"{'[dry] ' if dry_cards else ''}card{i+1}: {c}")

and the regex in recent_hooks() requires card1: to come straight after the timestamp, so [dry] card1: never matches. I checked both directions: the test line is in the file (grep finds it), and the parser returns zero rows for it. A guard you only tested in one direction is half-tested — I have shipped a filter that correctly blocked the bad case and silently ate a good one.

The general version

If your prompt contains an adjective describing the output, expect that adjective to show up in the output. "Punchy," "surprising," "counterintuitive," "engaging" — these read as instructions about tone, but they land as instructions about form.

And a stateless model cannot avoid repeating itself. Variety is not something you can ask for; it is something you have to give it the inputs for.

The tell is worth internalising: no repeated words, same repeated skeleton. Any per-token novelty check will pass that. You have to read a week of output in one sitting to see it, which is exactly what nobody does with a pipeline that has been quietly working for a month.

Your auto-captioner is measuring the wrong thing: a ratio that fired on a perfectly good clip

Youfu Hsu — Mon, 07 Sep 2026 04:54:19 +0000

I burn subtitles onto short AI-generated ad videos. The pipeline is small: generate the clip, transcribe it, write an .ass file, let ffmpeg burn it in.

It broke three times, and each break taught me something narrower and more useful than the last. The third one is the interesting one, because my fix was wrong in a way that looked completely right.

Break 1: hand-written timings never match generated speech

The first version used the timings from my own script. I wrote the lines, I knew roughly how long each one took to say, so I typed them in:

["A1_partner_snore", [
  [0.0, 3.2, "He snored through our entire honeymoon."],
  [3.2, 7.0, "I stopped hearing it on night one."]
]]

Every video was out of sync. Not slightly — by a second or more, drifting worse toward the end.

The reason is obvious in hindsight: I was timing my own reading of the line, and a generative video model has its own pacing. It pauses where it wants. It rushes short clauses. Nothing about my estimate was connected to the audio that actually exists in the file.

The only reliable source of timing is the audio itself:

import whisper
model = whisper.load_model("small")
res = model.transcribe(str(wav), language="en", fp16=False)
segs = [(s["start"], s["end"], s["text"].strip())
        for s in res["segments"] if s["text"].strip()]

Break 2: the transcript is the wrong text

Now the timing was right and the words were wrong.

Whisper mishears. In one clip "earplugs" came out as "Urplugs." Separately, the video model itself sometimes drops a word from the line it was handed.

So there are three versions of every sentence in play:

Source	Timing	Wording
My script	wrong	correct
Whisper transcript	correct	wrong, twice over
What I want on screen	correct	correct

Neither source is usable alone, and the fix is to stop treating them as competing answers.
Take timing from the audio, take text from the script.

def align(heard, script_lines):
    if len(heard) == len(script_lines):
        return [(heard[i][0], heard[i][1], script_lines[i])
                for i in range(len(script_lines))]
    # segment counts disagree: spread the script across the spoken span,
    # weighted by word count
    t0, t1 = heard[0][0], heard[-1][1]
    weights = [max(1, len(s.split())) for s in script_lines]
    total, out, t = sum(weights), [], t0
    for line, wt in zip(script_lines, weights):
        d = (t1 - t0) * wt / total
        out.append((t, t + d, line))
        t += d
    return out

This generalizes past captions. When two sources each have a known-good field and a known-bad field, joining on the good fields beats picking a winner.

Break 3: the frozen caption

Two clips came out with a single subtitle sitting on screen for about eight seconds, unmoving, while nothing was being said.

Those clips barely had dialogue. When speech is sparse, whisper happily extends a segment's end through the trailing silence. The segment is still correct as a transcript. It just isn't a caption cue.

I needed a guard that says this timing is untrustworthy, fall back to hand-written cues. Here was my first attempt:

# if any single segment covers more than 40% of the clip, distrust the timing
longest = max(e - s for s, e, _ in segs)
if longest / duration > 0.40:
    return None

It caught both broken clips. It also rejected a clip that was completely fine.

Why the ratio was the wrong quantity

The clip it wrongly rejected: 6.4 seconds inside a 15-second video, so 43% — over my threshold. But that segment was one long line of dialogue, delivered at 1.72 words per second, which is just... a person talking.

I spent a while wondering whether 40% should have been 50%, or 60%. That was the wrong question, and it's the part worth generalizing:

A long sentence taking a long time is not a defect. My ratio couldn't tell "someone spoke
for six seconds" apart from "someone spoke for one second and then there was five seconds of
nothing," because duration was in the numerator and nothing in the formula represented how
much was actually said.

No threshold fixes that. Under that measurement the two cases are genuinely identical, so any cutoff that catches one catches the other. Moving the number only chooses which failure you get.

What separates them is speech rate. Normal delivery runs about 2–3 words per second. Padding a segment with silence adds seconds without adding words, so the rate collapses:

MIN_WORDS_PER_SEC = 1.5

slowest = min((len(text.split()) / (e - s), s, e, text)
              for s, e, text in segs if e > s)
if slowest[0] < MIN_WORDS_PER_SEC:
    print(f"slowest segment is {slowest[0]:.2f} words/sec "
          f"({slowest[2] - slowest[1]:.1f}s of audio) "
          f"- silence merged into speech, falling back to manual cues")
    return None

Same shape of check, still one threshold, completely different behaviour — because the denominator now normalizes by the thing that actually causes the variance.

Clip	Longest segment	% of clip	Words/sec	Ratio guard	Rate guard
Long single line (good)	6.4s	43%	1.72	❌ rejected	✅ accepted
Sparse dialogue (broken)	8.1s	54%	0.49	✅ rejected	✅ rejected

The heuristic that survives is the one whose units mean something. "Fraction of clip" is a number about the clip. "Words per second" is a number about speech — and speech was what I was trying to judge all along.

The habit that caught my bad fix

I only found the false positive because of a rule I now apply to every guard I write:

Run the regression in both directions. The bad input must be rejected and a
known-good input must still be accepted.

Testing only the bug you just saw is how you ship a fix that breaks something adjacent. My ratio guard passed the test I wrote for it — it rejected the broken clip perfectly. The clip it silently damaged was one I wasn't looking at, because it had never been broken.

That's the same failure mode as writing your own test fixtures and being pleased when they all pass. If the input and the expected output come out of the same head at the same moment, agreement between them proves nothing. Run the real corpus. Look at what changed for the items you didn't touch.

Takeaways

Never time generated speech by hand. Transcribe it.
Timing and wording can come from different sources. Join them; don't pick a winner.
When a heuristic uses a proportion, interrogate the denominator. If it doesn't represent the mechanism you're trying to detect, no threshold will save you.
Regression-test in both directions, on real inputs, before and after.

Number 3 is the one I keep re-learning. A badly tuned threshold is loud and easy to fix. A well-tuned threshold on the wrong quantity looks like a working system, right up until it quietly throws away something good.

I write about running unattended automation and AI pipelines in production, including the
parts where the fix turns out to be the bug. The full system is the Claude Code Automation Playbook.

Cost doesn't set price: how I got a product decision wrong 4 times in one day

Youfu Hsu — Mon, 07 Sep 2026 04:52:19 +0000

I spent a day letting an AI agent research a product decision for me. It changed its recommendation four times. Every reversal was caused by the same mistake, and I didn't see it until the fourth one.

The mistake is embarrassingly basic: I was estimating the selling price from the cost.

Here's how that plays out when you're moving fast, and the three data sources that fixed it — all free, all public.

The setup

I was screening physical products to advertise. Two questions matter:

Is anyone making money in this category? (demand)
Could we make money in it? (unit economics)

I built a gate for question 1 first: scrape the ad library, count how many distinct sellers have been running ads for 30+ days, filter for keyword relevance. Ads that run for a year are a good proxy for "this is profitable for someone."

That gate worked. Then I needed selling prices for question 2, and I didn't have them. So I did the natural thing: took the supplier cost and multiplied by a typical retail markup.

That single shortcut caused every reversal below.

Reversal 1 → 2: the tax code moved

First recommendation: run the test in the US and UK.

Then I checked import rules properly. The US de minimis exemption — which let sub-$800 parcels enter duty-free — ended for all countries in August 2025. Small parcels from China now pay duty plus a flat customs entry fee of roughly $17 per parcel.

That flat fee is the killer, and it took me a while to see why:

Product	Goods cost	Duty (37.5%)	Flat entry fee	Fee as % of goods
A	$4.95	$1.86	$17.00	344%
B	$14.09	$5.28	$17.00	121%
C	$24.80	$9.30	$17.00	69%

The percentage duty is small — it's charged on the cheap goods value. The flat fee doesn't care what the item costs, so it lands hardest on exactly the low-price items that direct-to-consumer import was built on.

The UK still has its £135 relief (until October 2028). So: UK, not US.

Fine. Real finding. But notice it's about landed cost — still the cost side.

Reversal 2 → 3: I trusted stale data

Second recommendation: three specific products, ranked.

Except two of them had been scanned before I'd deployed a relevance filter on the ad scraper. Without it, the scraper counted every ad the keyword search returned — including romance novels and migraine coaches that happened to contain the word "ice" or "face."

I rescanned with the filter on. Two of my top picks turned out to be red oceans (3,000 and 6,500 active ads). The "sellers" in the old data included what were obviously personal accounts and coupon-spam pages, not brands.

Lesson, cheap version: when you fix a data collection bug, the old rows don't fix themselves. Write the backfill. I now have a rescan --stale mode that finds every row collected before the fix and re-runs it.

Reversal 3 → 4: the estimate was doing all the work

Third recommendation, with real conviction this time: one product, clean on both gates, priced at $102, break-even ROAS under 2.

Then I went to look at what competitors actually charge — and the leading brand in that category sells the equivalent product for $32.99.

I had computed $102 from supplier_cost × 3. There is no world in which that product sells for $102. At $33, after VAT and payment fees, the ad budget per order is about four dollars. It's not a marginal business, it's an impossible one.

That was reversal four, and it finally made the pattern visible.

The actual lesson

Cost does not set price. The market sets price. Cost only tells you what's left.

Look at what my markup assumption implied versus reality:

Product	Supplier cost	My estimate (×3)	What the leader actually charges	Real multiple
Sleep mask	$6.95	$21	$127 (catalog median)	18×
Bidet attachment	$34	$102	$33	1×

One product supports an 18× markup. Another supports 1×. Same formula, opposite errors — and no amount of care about the cost side would have caught either.

The fix isn't a better multiplier. It's to stop estimating a number you can go and measure.

Three free ways to measure the real price

1. Shopify's `/products.json` is public

Most DTC brands run Shopify, and Shopify exposes the full catalog as JSON with no auth:

https://<domain>/products.json?limit=250

You get every product, every variant, every price:

import json, urllib.request, statistics

req = urllib.request.Request(f"https://{domain}/products.json?limit=250",
                             headers={"User-Agent": "Mozilla/5.0"})
products = json.load(urllib.request.urlopen(req))["products"]

prices = [float(v["price"])
          for p in products
          for v in p.get("variants", [])
          if v.get("price") and float(v["price"]) > 0]

print(len(products), "products, median", statistics.median(prices))

That five-line query is what ended reversal 4. It's also what showed me something I'd have missed entirely: the category leader has 95 SKUs, and most of them are consumables — dryer sheets, cleaning solution, refills. They're not selling one gadget. They're selling a razor-and-blades business, which is why they can afford customer acquisition I can't.

2. Where do you get the domains?

From the ad copy. Advertisers put their own URL in the ad text, so a regex over scraped ad bodies gives you the competitor list for free:

DOMAIN = re.compile(r"\b((?:[a-z0-9][a-z0-9-]*\.)+(?:com|co\.uk|shop|store))\b", re.I)

Filter out the obvious noise (facebook.com, amazon.com, your ad platform) and rank by frequency — an advertiser's own domain repeats across their ads.

3. Traffic tells you if the category is growing or dying

Free-tier traffic estimators (3 lookups/day is plenty) give month-over-month change and channel mix. Two competitors, same day:

	Leader A	Leader B
Monthly visits	~232K	~52K
MoM change	+38.6%	−10.9%
Top channel	Paid social 32%	Paid search 37%
Bounce / pages / duration	42.9% / 3.18 / 2m07s	74.8% / 1.82 / 31s

Two things fell out of that table that nothing else told me:

The category I'd ranked first on search-trend data had a shrinking leader. Search volume and actual purchase traffic disagreed, and purchase traffic is closer to the money.
Leader B's biggest channel is paid search, not paid social. I'd been studying the wrong battlefield for that product.

What I changed in the pipeline

The screening gate now refuses to be confident on estimated prices:

if retail_price is None and score >= PASS_THRESHOLD:
    verdict = "WATCH"   # never PASS on an estimate
    reasons.append("price is estimated, not measured — go fetch the real one")

That one rule would have prevented three of the four reversals. Not because the estimate was badly calibrated, but because an estimate and a measurement should never be allowed to look the same downstream. Once $102 (estimated) becomes just 102 in a table, every decision after it inherits a confidence the number never had.

If you're building anything that ranks options, tag every input with how you got it. The ones you guessed are the ones that will move.

Exit code 0 is a lie: 7 ways my unattended automation silently did nothing

Youfu Hsu — Sun, 06 Sep 2026 03:44:16 +0000

I run about thirty scheduled jobs on a single Windows box. Some are scrapers, some generate content, some are trading bots, some just check that the other jobs are alive. Most of them were written and are maintained by an AI coding agent that I let run unattended.

Over three months, every one of the failures below reported success. The scheduler said LastTaskResult = 0. The logs looked fine or didn't exist. And nothing had happened.

If you only take one thing from this post: stop checking exit codes, start checking artifacts. I'll get to why at the end. First, the seven ways I got lied to.

1. The wrapper that always returns 0

To stop console windows flashing on my desktop every few minutes, I wrapped each scheduled task in a tiny VBScript launcher:

Set WshShell = CreateObject("WScript.Shell")
WshShell.Run "cmd /c ""python job.py >> job.log 2>&1""", 0, True

0 hides the window. True waits for completion. I assumed True also meant the exit code came back. It does not. WshShell.Run used as a statement discards the return value, so wscript.exe exits 0 no matter what the child did.

I found this because a content pipeline had been dead for five days while the scheduler reported green every single day.

The fix is to call Run as a function and pass the value out:

Set WshShell = CreateObject("WScript.Shell")
exitCode = WshShell.Run("cmd /c ""python job.py >> job.log 2>&1""", 0, True)
WScript.Quit(exitCode)

Note the parentheses — required when you're taking a return value. After fixing this across
17 launchers, one task showed a non-zero result for the first time in its life. It had
been failing for weeks.

2. The last line of your batch file overwrites the exit code

Fixed the launcher, still got false greens. The next layer down was a .cmd shim:

node pipeline.js >> run.log 2>&1
echo [done] exit code %errorlevel% >> run.log

That echo is the last command, echo always succeeds, so the batch file returns its exit code — zero — regardless of what node did. The log even contained the correct non-zero errorlevel. It just never made it out.

node pipeline.js >> run.log 2>&1
set NODE_EXIT=%errorlevel%
echo [done] exit code %NODE_EXIT% >> run.log
exit /b %NODE_EXIT%

Debugging heuristic: when you fix one layer and still get false greens, assume there's
another layer. Mine was three deep: scheduler → vbs → cmd → python.

3. No console, no stdout, no error, no service

A Flask service was set to start at login through a VBS launcher using pythonw.exe (the GUI-subsystem Python with no console). The script began with a routine encoding guard:

if sys.stdout.encoding != "utf-8":
    sys.stdout.reconfigure(encoding="utf-8")

Under pythonw, sys.stdout isn't a usable stream. That line raised, uncaught, before the server ever bound its port. No console existed to print the traceback to, so there was no error anywhere. The visible symptom was "the port isn't open after reboot," which sends you straight to firewall and networking — the wrong place entirely.

Fix: don't use pythonw. Use regular python.exe with output redirected to a file, so
stdout is a real stream:

exitCode = WshShell.Run("cmd /c ""python.exe app.py >> app.log 2>&1""", 0, True)

Heuristic: if a background service won't come up, check tasklist for the process
first. If it isn't there, the program died on startup — it's not a network problem.

4. Two scheduler defaults that quietly kill laptop jobs

Windows Task Scheduler ships with defaults that are reasonable for a desktop and lethal on a laptop:

Setting	Default	What it does to you
`DisallowStartIfOnBatteries`	true	Job doesn't run at all when unplugged. No error.
`StopIfGoingOnBatteries`	true	Job dies mid-run if you unplug.
`StopOnIdleEnd`	true	Job is killed when you touch the machine
`StartWhenAvailable`	false	A missed run is skipped, not retried

That third row cost me the most. A job would start on schedule, I'd sit down and move the mouse, idle state would end, and the scheduler would terminate the task mid-flight. The result code was "terminated," which is easy to misread as a crash in your own code.

Every new task I create now gets all four flipped:

$s = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries `
     -DontStopIfGoingOnBatteries -StartWhenAvailable -MultipleInstances IgnoreNew
Set-ScheduledTask -TaskName $name -Settings $s
$t = Get-ScheduledTask -TaskName $name
$t.Settings.IdleSettings.StopOnIdleEnd = $false
Set-ScheduledTask -TaskName $name -Settings $t.Settings

StopOnIdleEnd isn't exposed on New-ScheduledTaskSettingsSet, which is why it needs the second, uglier step — and why it's the one people miss.

5. A comment warning about a trap became the trap

This one is my favourite, because it's so stupid.

A batch file had a comment reminding future-me about a previous bug:

rem 坑備忘：exit /b %errorlevel% 必留

(It's in Chinese — "trap memo: exit /b %errorlevel% must stay" — which matters, because that's what makes it multi-byte.)

The file is UTF-8. cmd.exe decodes batch files using the system codepage, which on this machine is CP950, not UTF-8. Multi-byte characters get re-paired at the wrong boundaries, the comment gets truncated partway through, and the remainder of the line is executed as a command.

Every morning at 08:30 a console popped up: 'errorlevel' is not recognized as an internal or external command. The pipeline itself succeeded. The exit code was 0. The only symptom was a comment's corpse trying to run.

Rule I follow now: comments in .cmd/.bat files that run unattended are ASCII-only.
If you truly need non-ASCII, chcp 65001 >nul must be the first line — it can't rescue anything above itself.

The same class of bug bites .vbs (WSH reads it as ANSI — save as UTF-16 LE if it contains non-ASCII) and .ps1 (non-ASCII comments can desync the parser so badly that source lines end up inside string variables).

6. Your OS silently blocks the CLI tool you pip-installed

A publishing job died on:

OSError: [WinError 4551] This file is blocked by application control policy

It was calling yt-dlp via subprocess. Windows Smart App Control had decided that this particular unsigned, freshly-updated standalone .exe had no reputation yet and blocked it. The queue silently backed up for a day.

The fix is not to weaken your security policy. It's to stop invoking the standalone executable at all:

subprocess.run([sys.executable, "-m", "yt_dlp", ...])   # goes through trusted python.exe

Most pip-installed CLI tools support -m invocation. Prefer it in anything unattended.

7. The audit script you don't have

Every failure above shares a property: the status channel said one thing and reality said another. So I stopped trusting the status channel.

I now run two scripts, and they answer deliberately different questions:

Did it run, and what happened? Enumerate every scheduled task, resolve where its log actually goes (mine hide behind set LOG= variables, nested vbs→cmd calls, and relative redirects), and read the tail.
Did anything come out? For each job, define the artifact it's supposed to produce and the maximum age that's acceptable. Anything staler than that is flagged, regardless of what the exit code says.

The second one is the important one. It's how I found out a video pipeline had been running "successfully" every day for weeks while producing zero files.

CHECKS = [
    # (name, artifact glob, max age in hours)
    ("daily-report", r"C:\...\memory\daily_*.md", 26),
    ("product-radar", r"C:\...\data\radar_*.csv", 26),
    ("video-pipeline", r"C:\...\videos\*.mp4", 26),
]

Fifteen lines of config, and it catches an entire class of failure that no amount of exit-code checking will.

The pattern

Six of these seven are the same bug wearing different clothes: a layer between you and the work reported on itself instead of on the work. The wrapper reported on the wrapper. The batch file reported on its last echo. The scheduler reported that it launched something, not that the something did anything.

So the habit that actually protects you isn't better error handling. It's:

Define, in advance, the observable thing that proves the job did its work.
Then check for that, on a schedule, and alert when it goes stale.

Exit codes tell you a process ended. Artifacts tell you a job happened. Only one of those is what you actually care about at 3am when you're asleep and the machine is working.

I package the full version of this — 22 documented traps, the two audit scripts, wrapper
templates, and the memory system that stops an AI agent re-learning all of it every session — as the Claude Code Automation Playbook. But everything above is standalone; take it and go fix your own false greens.