DEV Community: Lily

4 Days, 3 Wasted Calls Per Run: My Retry Loop Mistook a Quota-Limit Message for a 'Too Short' Article

Lily — Mon, 21 Sep 2026 05:00:05 +0000

Last time I wrote about using a probe to decide which jobs to stop first when the quota runs dry. This post is about the problem sitting right next to it: my script could tell when the quota was exhausted, but it treated the limit message as if it were real article body text, then kept retrying a failure that no rewrite could ever fix. From September 13 to 16, every run burned 3 claude -p calls to produce nothing. After the fix, it burns 1 and bails out.

The problem: 4 straight days (9/13–9/16) of 3 failures per run

note2-daily-stock.sh auto-generates product review articles for note2 (bokumolily) using claude -p. For each product it calls claude -p up to 3 passes to produce the body. Each pass is judged by a quality check (article_quality_ok) with criteria like "does the character count exceed MIN_CHARS (7,000)?" and "are there at least 2 affiliate links?"

For 4 days, 9/13 through 9/16, this script failed 3 times in a row with the exact same symptom. The cause: when claude -p hits the Max plan's limit, it returns a one-line limit-reached message (no TITLE: line), and the script treated that as "an ordinary generation failure where the body is too short." The comment in the actual code records exactly what happened:

  # 2026-09-16: claude -pがクォータ上限メッセージ(1行・TITLE:行なし)を返すと
  # 本文が空のまま「文字数不足」として3回とも空振りし、上限中に3回分のclaude呼び出しを浪費していた
  # (note2-daily 9/13-9/16 4日連続で同一症状を実測)。上限メッセージは即検出して1回で中断する。

(note2-daily-stock.sh:565-567)

While the limit is in effect, no amount of rewriting will produce a body. Yet article_quality_ok only ever said "quality NG reason: not enough characters," so the script concluded "maybe one more pass will fix it" and threw away 2 more pointless claude -p calls during the limit window.

Note: A retry loop needs more than the monolithic fact that "it failed." If you don't distinguish "a failure a rewrite can fix" from "a failure a rewrite cannot fix," you'll keep walking into the same wall.

The fix: grep for the limit message and abort after 1 pass

The fix is to grep for the limit-message pattern immediately after each pass's generation, and if it matches, abort right there instead of spending the remaining passes.

  for pass in 1 2 3; do
    RAW=$(generate_article_raw "$PRODUCT_JSON" "$IMG_COUNT")
    if printf '%s' "$RAW" | grep -qiE "hit your (weekly |5-hour |usage |session )?limit|usage limit reached|rate limit|resets? (at |in |on )?[0-9]{1,2}(:[0-9]{2})? *[ap]m"; then
      QUOTA_HIT=1
      log "ABORT: quota-message検出のため中断 (pass $pass) $SLUG raw先頭=[$(printf '%s' "$RAW" | tr '\n' ' ' | cut -c1-160)]"
      break
    fi
    write_article_file "$RAW" "$OUT"
    ...
  done

(note2-daily-stock.sh:563-572, excerpt)

When QUOTA_HIT is set, the script deletes the half-written output file and any images downloaded along the way, notifies Discord, and exits immediately. Instead of 3 calls, it consumes exactly 1 at that point and retreats.

  if [ "$QUOTA_HIT" -eq 1 ]; then
    [ -e "$OUT" ] && rm -f "$OUT"
    rm -f "$ROOT/${IMG_BASENAME}-p"*.jpg
    discord_notify "⚠️ note2-daily: claudeクォータ上限のため中断（3回消費せず中止）: $ITEM_NAME"
    exit 1
  fi

(note2-daily-stock.sh:590-595)

"Not enough characters" and "limit reached" look alike on the surface (the body is nearly empty), but the correct response is completely different. The former might be fixed by a rewrite; the latter won't be fixed no matter how many times you retry until the reset time. Separating those two cases up front with a single grep is the entire fix.

The pitfall I stepped in: the sibling scripts still don't have it

There are 3 other scripts that share the same shape as note2-daily-stock.sh — "have claude -p write the body, retry if the quality check fails": article-daily-stock.sh (the script that writes this very Zenn article series), maker-daily-stock.sh, and series-daily-stock.sh. I grepped all 3, and none of them contained the limit-message detection pattern.

$ grep -n "hit your\|usage limit\|QUOTA_HIT\|rate limit" \
    article-daily-stock.sh maker-daily-stock.sh series-daily-stock.sh
(該当なし)

In particular, run_pass() in series-daily-stock.sh has almost exactly the same structure that note2 tripped over.

run_pass(){ # $1=prompt $2=outfile $3=label
  local prompt="$1" outfile="$2" label="$3" attempt wait
  for attempt in 1 2 3; do
    rm -f "$outfile"
    timeout 600 "$CLAUDE" ... -p "$prompt" ... > "$outfile" ...
    if [ -s "$outfile" ]; then
      ...
    fi
    log "WARN: pass $label 空出力(attempt $attempt/3)"
    if [ "$attempt" -lt 3 ]; then
      wait=$((attempt * 30))
      sleep "$wait"
    fi
  done

(series-daily-stock.sh:254-279, excerpt)

Waiting 30 seconds, then 60 seconds, on "empty output (attempt N/3)" and hammering the same call 3 times for the same reason is essentially the same structure note2 hit on 9/13–9/16. If this script runs during a limit window, it should waste calls in exactly the same way — and it hasn't been fixed yet.

maker-daily-stock.sh only has a single generation pass, so it doesn't fail 3 in a row, but the same dates show up in its comments too.

  # 失敗時もキューを1件進める。進めないと同じ題材が先頭に居座り続け、翌日以降も
  # 同じ理由で落ち続ける（2026-09-13〜16 に同一題材が7回連続で失敗した実測）。

(maker-daily-stock.sh:412-413)

In other words, the Max plan limit on 9/13–9/16 didn't just hit note2 — it dragged the maker pipeline down with it. On the maker side, the underlying problem of "misdiagnosing the limit message as an incomplete body" is still there, exactly as in note2; it's being held together by a different band-aid: "even on failure, at least advance the topic queue." The same root cause has a different emergency patch in each script, applied whenever someone happened to notice.

I confused a one-line limit message with a "body too short" quality failure → grep for it right after generation and abort before the quality check
Judging retries only by "empty/non-empty" or "character count" can't tell failure types apart → branch on the failure reason (retry the fixable ones, exit immediately on the unfixable ones)
Even when one limit drags down multiple pipelines at once, the fix doesn't propagate → note2 detects the cause, maker advances the queue, article/series are untouched — the responses are still all over the place
Fixing one script doesn't automatically carry over to sibling scripts that copied the same structure → don't assume "it's fixed" until you've confirmed it with grep

Summary

For 4 days (9/13–9/16), note2-daily-stock.sh misdiagnosed the limit-reached message as "not enough characters" and burned all 3 pointless claude -p calls during the limit window
The fix greps for the limit-message pattern immediately after generation and, on a match, aborts after 1 pass, ahead of the quality check
During the same 4 days, maker-daily-stock.sh also failed 7 times in a row on the same topic — the limit hit multiple pipelines simultaneously. But maker's fix is only "advance the queue," and the underlying misdiagnosis remains unfixed there as well as in article-daily-stock.sh and series-daily-stock.sh
A retry loop that doesn't distinguish failure types (fixable vs. unfixable) will step in the same hole every time. A lesson learned in one place only pays off once it's propagated to other scripts with a similar structure

Next time I plan to write about whether that propagation itself can be automated — a mechanism to mechanically carry a fix found in one script over to its "structurally similar siblings."

How do your retry loops tell a fixable failure from one that no retry will ever fix?

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

One Line of mv Kills the 4x Duplicate Run After Recovery: Making launchd Queue Processing Idempotent

Lily — Mon, 21 Sep 2026 00:00:06 +0000

Six months ago my side business made ¥0 a month. This month it's ¥1.2M, running on an autonomous setup built with Claude Code. And here's the thing about automation: the moment it breaks is exactly when the sloppiness in your design gets exposed.

Why This Design Works

The Fork Between "Fixing by Habit" and "Fixing the Environment"

When automation breaks, the reflex is to reach for a procedural fix: "next time I'll do this instead." Check the logs more often. Retry by hand. Space things out a bit more. All of these are bandages meant to shrink the wound the next time the problem recurs. They don't remove the cause, so once the same conditions line up, it will happen again.

After six months of running an automated system, one thing I'm certain of: a scheduler combined with a shared directory will always collide unless you explicitly implement mutual exclusion. Not "it might collide if you're unlucky," but "it will collide whenever the conditions for collision line up." This isn't a probability problem; it's a structural one.

On September 17, 2026, Codex was down for an entire day with usage_limit_exceeded. Under ~/.codex/sessions/2026/09/17/ there were 54 sessions piled up, every one of them terminated with a limit error. Throughout that time, three workflows—note-autolike, ai-portraits-fragments, and social-autolike—kept firing jobs, and unprocessed JSON files accumulated in ~/dev/note-autolike/done/. Measured the next morning at 05:20, the JSON files flagged needs_imagegen_thumbnail had swelled from 43 to 57. Fourteen new files had stacked up, all of them left untouched.

The instant Codex came back, five launchd lanes fired at once. Each lane's pickup script grabs the JSON files under done/ with ls or a glob and starts processing. With no mutual exclusion, multiple processes look at the same JSON at the same time. The result: the funnel-pm lane processed the same JSON 4 times, one minute apart. As the numbers show—8 of 11 files were hit 2 or more times—this wasn't a freak accident. It was an inevitable consequence of the design.

launchd Doesn't Know About "Fighting Over Files"

Look at StartCalendarInterval in com.lily.codex-note-funnel.plist and you'll see two entries: 10:40 and 16:40.

<key>StartCalendarInterval</key>
<array>
    <dict>
        <key>Hour</key>
        <integer>10</integer>
        <key>Minute</key>
        <integer>40</integer>
    </dict>
    <dict>
        <key>Hour</key>
        <integer>16</integer>
        <key>Minute</key>
        <integer>40</integer>
    </dict>
</array>

launchd simply invokes run-codex-funnel.sh according to this schedule. Whether the previous run finished, or whether another lane is currently working on the same file—launchd has no interest in any of it. And that's correct, by-design behavior. launchd is a scheduler, not a mutex. Mutual exclusion over a file queue is the application layer's responsibility—in other words, it's a feature the pickup script itself has to implement.

Backlog Plus Simultaneous Recovery Is What Surfaces the Race

In normal operation, each lane's timing is slightly offset, which naturally keeps collisions rare. Target JSON files also arrive one at a time in sequence, so there are few opportunities for multiple lanes to see the same file simultaneously.

But a long outage like Codex's usage_limit_exceeded changes the situation completely. All lanes start at once with 57 JSON files backed up from the outage period.

通常運転:
  レーンA → job_001 処理 → 完了
  レーンB → job_002 処理 → 完了  （タイミングがズレており競合しない）

Codex 復帰直後:
  レーンA ─┐
  レーンB ─┤──→ job_001 を同時取得 → 4連射
  funnel-pm ─┘
  （57本が溜まっており、全レーンが先頭ファイルに殺到）

A procedural fix like "adjust the intervals" can't handle this "burst release" pattern. No matter how tightly you tune the intervals, the instant after recovery, every lane will fire simultaneously.

Why a Lock File Isn't Enough

"Just create a lock file" is the natural idea. But look closely at the implementation and there's a problem.

# NG: check-then-act は原子的でない
LOCKFILE=~/dev/note-autolike/.processing.lock
if [ ! -f "$LOCKFILE" ]; then
    touch "$LOCKFILE"
    process_file "$json"
    rm "$LOCKFILE"
fi

Between the if [ ! -f ] check and the touch, there's a "Time of Check to Time of Use" (TOCTOU) window where another process can slip in. Two processes both observe "no file" and both execute touch—that's the race. Under high load and high frequency, this window really does open.

On top of that, if the process crashes mid-run, the lock file stays behind. Every subsequent run sees the lock and exits immediately, so processing stalls completely. You need a separate crash-recovery procedure, and operational cost goes up.

File locking via the flock command is another option, but since file descriptors aren't inherited each time launchd spawns a new process, it doesn't work for exclusion between lanes.

The Guard Philosophy Shown by autolike-plist-reconcile.sh

The existing ~/.claude/scripts/autolike-plist-reconcile.sh contains a guard built with the same concern in mind. That script does a reload when the AUTOLIKE_TIMEOUT_SEC value in the plist has drifted from the value launchd has loaded, but the core is this code:

pid=$(launchctl list | awk -v l="$L" '$3==l{print $1}')
if [ -n "$pid" ] && [ "$pid" != "-" ]; then
    echo "[$(date '+%F %T')] $L 実行中(pid=$pid) のため見送り ($have -> $want)" >>"$LOG"
    continue
fi
launchctl bootout "$D/$L" 2>/dev/null

The comment at the top of the script says that booting out a running job wipes out that run's likes entirely, so it must always wait. It fetches the PID via launchctl list and skips the bootout unless the value is - (not running). Before starting work, check "is someone else already processing this?"—and if so, exit immediately. That's the same idea as the mv pattern in this article.

That said, a PID check has limits. You can't reduce to zero the chance that a new process starts between the "check" and the "bootout." In the context where this script runs, that's acceptable, but for claiming exclusive ownership of a file queue, it's insufficient. You need a stronger atomic operation.

The Overall Flow

Architecture Diagram: From Race to Resolution

Codex usage_limit_exceeded から復帰
              │
              ▼
    ~/dev/note-autolike/done/ に 57 本が滞留
    ┌────────────────────────────────────────┐
    │  job_20260917_001.json                │
    │  job_20260917_002.json                │
    │  ...                                  │
    │  job_20260917_057.json                │
    └────────────────────────────────────────┘
              │
    5 レーンが launchd により一斉起動
    ┌────────────────────────────────────────┐
    │ com.lily.codex-note-funnel   (10:40)  │
    │ com.lily.autolike.note1               │
    │ com.lily.autolike.note2               │
    │ com.lily.autolike.funnel-pm  ← 4連射  │
    │ com.lily.autolike.social              │
    └────────────────────────────────────────┘
              │
    【修正前】各レーンが done/ を glob → 先頭ファイルを取得
              │  排他なし → 全レーンが job_001 を同時に read
              ↓
    funnel-pm が 1分差で同一 JSON を 4 回処理
    ─────────────────────────────────────────
    【修正後】mv による原子的所有権取得
    ┌────────────────────────────────────────┐
    │  mv done/job_001.json processing/     │
    │  ├─ 成功: 自プロセスのみが処理を継続   │
    │  └─ 失敗: 他プロセスが取得済み → exit 0│
    └────────────────────────────────────────┘
              │
    「1ファイル = 1プロセスのみ処理」が保証される

Why mv Is Atomic

On the same filesystem, mv executes a single POSIX rename(2) system call. Because the "rename this file" operation completes indivisibly at the kernel level, even if two processes simultaneously run mv done/job_001.json processing/job_001.json, exactly one succeeds and the other is guaranteed to fail. No window opens between "check" and "acquire."

This is the fundamental difference between a PID check and mv. With a PID check, "check" and "start processing" are separate operations. With mv, "check" and "acquire" are completed inside one system call.

# 2プロセスが同時に実行した場合の挙動
# プロセスA: mv done/job_001.json processing/job_001.json → 成功（終了コード 0）
# プロセスB: mv done/job_001.json processing/job_001.json → 失敗（No such file or directory, 終了コード 1）

Wiring It into the Pickup Script

Here's the implementation to insert at the top of run-codex-funnel.sh.

#!/bin/bash
set -uo pipefail

DONE_DIR=~/dev/note-autolike/done
PROCESSING_DIR=~/dev/note-autolike/processing
mkdir -p "$PROCESSING_DIR"

# done/ から処理対象を1本選ぶ
TARGET=$(ls "$DONE_DIR"/*.json 2>/dev/null | head -1)
[ -z "$TARGET" ] && exit 0          # 対象なし → 正常終了

BASENAME=$(basename "$TARGET")

# アトミックに所有権を取得。失敗 = 他プロセスが取得済み
mv "$TARGET" "$PROCESSING_DIR/$BASENAME" 2>/dev/null || exit 0

WORKING="$PROCESSING_DIR/$BASENAME"
# ここから先は自プロセスだけが処理する権利を持つ

There's effectively only one situation in which mv fails: another process has already mv'd the file and it no longer exists. That is precisely the race condition itself, so exiting immediately with exit 0 is the correct response. You don't even need to write an error log.

A cross-device mv (spanning different mount points) is not atomic, so be careful—but in this setup, both done/ and processing/ live under ~/dev/note-autolike/, which guarantees the same filesystem.

Connecting with com.lily.codex-note-funnel.plist

The ProgramArguments in com.lily.codex-note-funnel.plist are structured like this:

<key>ProgramArguments</key>
<array>
    <string>~/.claude/scripts/claude-quota-guard.py</string>
    <string>--job</string>
    <string>com.lily.codex-note-funnel</string>
    <string>--priority</string>
    <string>--</string>
    <string>~/dev/note-autolike/run-codex-funnel.sh</string>
</array>

claude-quota-guard.py runs first as a quota-control wrapper, and if there's headroom, it launches run-codex-funnel.sh. Since the pickup logic lives inside run-codex-funnel.sh, inserting the mv ownership claim into the first few lines of the script is all it takes. No plist changes required.

launchd will keep calling run-codex-funnel.sh at 10:40 and 16:40. Even with 57 JSON files backed up right after Codex recovers from usage_limit_exceeded, every time run-codex-funnel.sh starts it repeats the same behavior: "first claim one file with mv; if that fails, exit immediately." Even if five processes, including the funnel-pm lane, run at the same time, exactly one process handles each file.

Implementation Details

The Full Script: The Order for Stacking Defensive Layers

The first half showed only the single mv line, but to actually run it in production you need layers before and after. Looking at ProgramArguments in com.lily.codex-note-funnel.plist, claude-quota-guard.py runs as the leading wrapper, and then run-codex-funnel.sh is called. The full script looks like this:

#!/bin/bash
# launchd 環境の PATH は最小限のため先頭で上書きする
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
export PATH

set -uo pipefail

DONE_DIR=~/dev/note-autolike/done
PROCESSING_DIR=~/dev/note-autolike/processing
PROCESSED_DIR=~/dev/note-autolike/processed
LOG=~/dev/note-autolike/logs/codex-funnel.log

mkdir -p "$PROCESSING_DIR" "$PROCESSED_DIR"

# 起動時リカバリ: 前回クラッシュで processing/ に残ったファイルを done/ へ戻す
for stale in "$PROCESSING_DIR"/*.json; do
    [ -f "$stale" ] || break
    age=$(( $(date +%s) - $(stat -f %m "$stale") ))
    if [ "$age" -gt 3600 ]; then
        mv "$stale" "$DONE_DIR/" 2>/dev/null
        echo "[$(date '+%F %T')] recovered stale: $(basename "$stale")" >> "$LOG"
    fi
done

# done/ から処理対象を1本選ぶ
TARGET=$(ls "$DONE_DIR"/*.json 2>/dev/null | head -1)
[ -z "$TARGET" ] && exit 0

BASENAME=$(basename "$TARGET")

# ─── ここが核心 ─────────────────────────────────────────
# アトミックに所有権を取得。失敗 = 他プロセスが取得済み → 即退出
mv "$TARGET" "$PROCESSING_DIR/$BASENAME" 2>/dev/null || exit 0
# ────────────────────────────────────────────────────────

WORKING="$PROCESSING_DIR/$BASENAME"

# クラッシュ時に processing/ へファイルが残るのを防ぐ EXIT トラップ
# 正常完了時は processed/ へ移動、異常時は done/ へ差し戻す
_cleanup() {
    local code=$?
    if [ $code -eq 0 ]; then
        mv "$WORKING" "$PROCESSED_DIR/$BASENAME" 2>/dev/null
    else
        mv "$WORKING" "$DONE_DIR/$BASENAME" 2>/dev/null
        echo "[$(date '+%F %T')] ERROR exit=$code, returned: $BASENAME" >> "$LOG"
    fi
}
trap '_cleanup' EXIT

# ── ここから先は自プロセスだけが $WORKING を処理する ──
echo "[$(date '+%F %T')] processing: $BASENAME" >> "$LOG"
# ... 本体処理 ...

Why Override PATH at the Top

Look at EnvironmentVariables in com.lily.codex-note-funnel.plist: the key exists, but the content is an empty <dict/>.

<key>EnvironmentVariables</key>
<dict>
</dict>

When launchd starts a process, its PATH is only the minimal set /usr/bin:/bin:/usr/sbin:/sbin. If you echo $PATH in a terminal you'll see /usr/local/bin included, but that's added by .zshrc or the nvm initialization script. launchd doesn't run any such shell initialization. Tools installed in /usr/local/bin like jq, node, and python3 fail silently as "command not found" in scripts launched from a plist. The fix is one of two choices: write it in the plist's EnvironmentVariables, or hardcode it at the top of the script. The latter is more convenient because you can verify the script's behavior on its own, so I've standardized on export PATH at the top.

Why `set -uo pipefail` and `|| exit 0` Must Go Together

When set -uo pipefail is active, the whole script aborts the moment a command returns a non-zero exit code. mv returns exit code 1 when the source file doesn't exist. That maps exactly onto the normal race scenario of "another process already claimed it."

# NG: set -uo pipefail 環境では mv 失敗でスクリプトが非ゼロ終了する
mv "$TARGET" "$PROCESSING_DIR/$BASENAME" 2>/dev/null

# OK: 失敗を明示的に「正常な退出」に変換する
mv "$TARGET" "$PROCESSING_DIR/$BASENAME" 2>/dev/null || exit 0

2>/dev/null is needed as its counterpart. With || exit 0 alone, "No such file or directory" is printed to stderr. Since StandardErrorPath in com.lily.codex-note-funnel.plist points to ~/dev/note-autolike/logs/codex-funnel.error.log, every race piles a line that isn't actually an error into the error log. When you monitor logs, this becomes the cause of false alarms like "the error log is growing."

The "Double Check" Idiom Demonstrated by autolike-plist-reconcile.sh

The PID-check portion of autolike-plist-reconcile.sh has a subtlety that's easy to miss.

pid=$(launchctl list | awk -v l="$L" '$3==l{print $1}')
if [ -n "$pid" ] && [ "$pid" != "-" ]; then

The first column returned by launchctl list is either a PID or -. - means the job is registered but not currently running. With [ -n "$pid" ] alone, a pid of - would also be judged as "has a PID." That's why [ "$pid" != "-" ] is joined with AND.

This idiom of checking "the variable is non-empty" and "the value is meaningful" separately can be applied to the mv pattern too. If you want to add a defensive layer that verifies the WORKING file actually exists rather than looking only at the exit code of mv, write it like this:

mv "$TARGET" "$PROCESSING_DIR/$BASENAME" 2>/dev/null || exit 0
# 念のため: mv が成功したはずなのにファイルがない（同一FSでない等）
[ -f "$WORKING" ] || { echo "FATAL: mv succeeded but file missing" >> "$LOG"; exit 1; }

This extra check is normally never reached, but it makes the symptom explicit if you accidentally run a cross-device mv. In an environment with set -uo pipefail active, having the cause in the log beats dying silently—debugging afterward is dramatically faster.

Design Rationale for Startup Recovery

The step at the top of the script that "returns processing/ files older than one hour to done/" is a dead-letter countermeasure.

age=$(( $(date +%s) - $(stat -f %m "$stale") ))
if [ "$age" -gt 3600 ]; then
    mv "$stale" "$DONE_DIR/" 2>/dev/null
fi

If the process crashes after claiming ownership with mv, the file stays in processing/. The EXIT trap is designed to return it to done/, but the trap won't run in cases like a forced kill with signal 9. The symptom of "I check done/ the next morning and it's empty, yet nothing got processed" is caused by this pattern. By running recovery at startup, the file automatically re-enters the queue on the next launchd invocation. The threshold is 3600 seconds (one hour) so that legitimately long-running jobs don't get returned by mistake. Since StartCalendarInterval in com.lily.codex-note-funnel.plist has the two entries 10:40 and 16:40, the maximum schedule gap is six hours. One hour is a sufficient safety margin within that range.

Where I Got Stuck

Stuck #1: The Race Was Gone, but the Error Log Kept Growing

The day after implementing mv || exit 0, I looked at codex-funnel.error.log and the line count had gone from 1 to 32. The processed count was increasing normally. The 4x duplicates were gone. And yet the error log was growing.

At first I assumed "there must be an exception somewhere" and started debugging the main processing. After about 30 minutes of chasing it, I realized 2>/dev/null was missing. The code I first wrote was this:

mv "$TARGET" "$PROCESSING_DIR/$BASENAME" || exit 0

|| only handles the exit code. It doesn't stop output to stderr. launchd keeps writing stderr to StandardErrorPath. As a result, "No such file or directory" accumulated at every moment of contention.

The correct form is this:

mv "$TARGET" "$PROCESSING_DIR/$BASENAME" 2>/dev/null || exit 0

The cause was that I hadn't built the habit of always writing these two together. My preconception that "error log growing = bug in the main body" sent my debugging in the wrong direction. When the error log grows, the first thing to check is "am I silencing the normal failures?"—I should have made that order routine from the start.

Stuck #2: Files Kept Piling Up in processing/ While done/ Looked Empty Forever

This happened in the version before I implemented the EXIT trap. Codex crashed overnight several times in a row, and when I checked the next morning, done/ was empty but processing wasn't happening.

Tracing the symptom, five JSON files had accumulated in processing/. Each time the script started, it checked done/, found TARGET empty, and did exit 0. But the "files that should be processed" were stuck in processing/.

done/       ← 空（次の仕事はここに来る）
processing/ ← job_001.json, job_002.json ...  ← ゾンビ
processed/  ← 完了済み

This state does double damage. First, processing stops. Second, "the fact that it's stuck isn't visible from outside." Because done/ is empty, the automation looks healthy. But even when new files arrive in done/, nobody ever touches the zombies in processing/.

The fix combined two things: returning files via the EXIT trap and recovering stale processing/ files at startup. Either one alone is insufficient; to handle forced kills where the trap doesn't run, startup recovery is required. After implementing this, I only need to periodically ls -la the processing/ directory to confirm nothing is lingering, and this type of stall resolves itself by the next morning.

Stuck #3: Works When Run from the Terminal, Silently Does Nothing from launchd

This was the failure that burned the most time. Running run-codex-funnel.sh directly in the terminal processes JSON files normally. Via launchd, "processing: ..." never appears in codex-funnel.log. The error log is empty too. The script is starting, but doing nothing.

What I used to pin down the cause was reproducing the shell launchd starts as faithfully as possible.

env -i HOME="$HOME" PATH=/usr/bin:/bin:/usr/sbin:/sbin bash -l ~/dev/note-autolike/run-codex-funnel.sh

Running it with env -i to minimize the environment variables, jq: command not found appeared partway through. The main processing used jq to read the needs_imagegen_thumbnail flag from the JSON, but jq was installed at /usr/local/bin/jq. With set -uo pipefail active, the script exits non-zero the moment jq isn't found. The exit code should have been written to codex-funnel.error.log, but since the error message itself never went to stderr (no jq means no error text either), the log was empty.

Had I read that EnvironmentVariables in com.lily.codex-note-funnel.plist was empty, I would have noticed sooner. The cause was skipping verification on the assumption that "PATH is probably fine" without actually reading the file. Since then, writing PATH= at the top has been an absolute rule for every script run via launchd.

PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
export PATH

Whether this one line is present is now the first thing I check in code review.

Stuck #4: `ls *.json` Returned Exit Code 2 on "No Files" and Crashed the Script

When done/ is empty, ls "$DONE_DIR"/*.json fails shell glob expansion and returns exit code 2. In a set -uo pipefail environment, this causes the script to terminate abnormally.

# NG: done/ が空のとき ls が exit 2 → pipefail でスクリプト全体が落ちる
TARGET=$(ls "$DONE_DIR"/*.json | head -1)

# OK: 2>/dev/null でエラーを捨て、変数が空かどうかで判断する
TARGET=$(ls "$DONE_DIR"/*.json 2>/dev/null | head -1)
[ -z "$TARGET" ] && exit 0

This problem wouldn't occur if the design treated an empty done/ as "abnormal." But this setup is designed so that "in normal operation, done/ is empty more often than not," so empty is the normal path. To "treat empty as normal," you need a line that explicitly absorbs the command's failure.

Looking at how autolike-plist-reconcile.sh is written, there are three places where it uses continue to skip out of the loop.

want=$($PB -c "Print :EnvironmentVariables:AUTOLIKE_TIMEOUT_SEC" "$P" 2>/dev/null) || continue
have=$(launchctl print "$D/$L" 2>/dev/null | ...) | head -1)
[ -n "$have" ] || continue
[ "$want" = "$have" ] && continue

|| continue appears throughout. The decision "if we can't get this lane's info, skip it" is written as moving on to the next iteration without stopping on the error. This idea of "converting a failure into an instruction for the next step" is the same philosophy as mv ... || exit 0. Errors aren't something to leave in the log—they're a signal for the next action. Once I switched to this viewpoint, my scripts became dramatically clearer.

Stuck #5: It Took a Full Day to Notice `funnel-pm` Was Firing 4x

The last "stuck" is less about technology and more about observability. I only noticed the 4x duplicate on September 17 when I counted the files in processed/ at 05:20 the next morning, on the 18th. Looking at the log, there were four entries for the same JSON, one minute apart.

For the 17 hours the problem was happening, I knew nothing. The logs accumulate in ~/dev/note-autolike/logs/codex-funnel.log, but nobody monitors them in real time. The fact that "processing was duplicated" could only be detected from duplicate filenames in processed/.

I now use this duplicate-detection one-liner in my morning check:

ls ~/dev/note-autolike/processed/ | sed 's/_[0-9]*$//' | sort | uniq -d

It strips the timestamp portion of the filenames and checks for duplicate base names. If it returns anything, that day's processing had a double execution.

Structural improvement that drives duplicates to zero (the mv pattern) and, independently, monitoring that lets you notice duplicates happened—these are problems on different layers. Build a design that doesn't break, and also have a mechanism that lets you notice a break the same day rather than the next. In an environment where automation feeds real revenue, this two-tier approach is indispensable.

Pitfalls

Environment and Configuration

launchd does not inherit PATH. Running with an empty EnvironmentVariables block will get you stuck. EnvironmentVariables in com.lily.codex-note-funnel.plist is empty, and the effective PATH at launchd startup is only /usr/bin:/bin:/usr/sbin:/sbin. The codex installed via nvm is not on this PATH. Even though run-codex-funnel.sh has set -uo pipefail, in a "command not found" situation there are cases where -u reacts late (which codex returns an empty string). The fastest diagnosis is to start it manually with launchctl start com.lily.codex-note-funnel and check the logs.
Nice: 10 and LowPriorityIO: true conflict with burst processing. Both plist settings deprioritize the script's execution when the system is under heavy load. When you want to churn through 57 JSON files right after Codex recovers, Nice: 10 is a "yield CPU to other user operations" setting, so processing is delayed. "Run quietly in the background" and "process at full speed after recovery" are in conflict. Decide which to prioritize and write the intent in a comment. The current plist chooses the former.
The default ThrottleInterval depends on the launchd version. Accumulated non-zero exits can cause StartCalendarInterval scheduled starts to be skipped. This ties directly into failure #3 in part 2 (throttling on exit 1), but to add: if 10:40 is skipped due to throttling, the next scheduled run is 16:40, six hours later. The reliable diagnosis is to go back through days where nothing was written to codex-funnel.error.log (StandardErrorPath). If you consistently use exit 0, the default ThrottleInterval setting is fine, but if you're "leaving it to the default," record that intent in a plist comment or documentation.
An mv failure due to a missing claimed/ directory and an mv failure due to race avoidance both return the same exit 0. I touched on this in failure #5 of part 2, but let me restate it as the underlying design trade-off. Giving "normal race avoidance" and "abnormal environment misconfiguration" the same exit code prevents launchd's throttle, but delays discovery of problems. Since you're paying the price of this silence, external monitoring is mandatory.

Traps Specific to the mv Claim Pattern

Cross-device mv is not atomic. POSIX rename(2) is serialized by the kernel only "on the same filesystem." If queue/ and claimed/ span different mount points (e.g., NFS, Docker volumes, external drives), the operation decomposes internally into copy + delete, and a TOCTOU window opens. This setup has both directories under ~/dev/note-autolike/, so there's no issue, but whenever you change the directory layout, verify the same filesystem with df -h.
Filename sort order becomes unstable when files are created in the same second. The lexicographic sort in ls -1 "$QUEUE_DIR"/*.json | sort | head -1 assumes the timestamp portions of the filenames differ. When batch submission creates multiple files within one second, same-name prefixes collide, and five lanes grab different files as the "first." As a result, no claim contention occurs, all lanes process simultaneously, and five Codex sessions start at once. The workaround is to include epoch seconds + PID suffix in the filename: job_$(date +%s)_$$_$(uuidgen | cut -d- -f1).json.
Leftover files in claimed/ become a silent dead-letter queue. If processing crashes midway, the file stays in claimed/. The next launchd start looks at queue/, so files in claimed/ are never picked up again. In fact, after Codex recovered on September 17, 2026, there was a day when several files were stuck in claimed/. Since adding monitoring that alerts when a file in claimed/ has gone more than an hour without an update, the lag before noticing this state has dropped to zero.
The claude-quota-guard.py fd-inheritance issue remains even after the mv claim. The first entry in ProgramArguments in com.lily.codex-note-funnel.plist is ~/.claude/scripts/claude-quota-guard.py. While this Python wrapper launches run-codex-funnel.sh via subprocess.Popen, the job appears "running" to launchd. The PID check in autolike-plist-reconcile.sh (verifying via launchctl list | awk -v l="$L" '$3==l{print $1}' that the PID is not -), which skips reload on the judgment "codex-funnel is running," correctly captures this "wrapper is alive" state as well. The mv claim eliminated the race, but you need to stay aware of how PIDs are read given that the wrapper launch is a precondition.
The glob in reconcile.sh excludes codex-note-funnel. autolike-plist-reconcile.sh targets com.lily.autolike.*.plist. com.lily.codex-note-funnel.plist is a separate file and doesn't match this glob. Since the automatic timeout-value reload that reconcile performs doesn't apply to codex-funnel, changing codex-funnel's timeout setting requires a manual launchctl bootout + bootstrap. If this is intentional, state it explicitly in a comment so that six months from now you won't be confused.

Operations and Monitoring Traps

No log rotation configured for codex-funnel.log. The plist's StandardOutPath points to ~/dev/note-autolike/logs/codex-funnel.log. launchd doesn't rotate logs. With dozens of lines per run accumulating continuously, it reaches tens of MB in a year. You need explicit rotation via newsyslog or logrotate, or line-count cap management inside the script.
Without a mechanism to automatically detect "log not updating," you won't notice a stall for a week. I've separately added monitoring that fires an alert if the update timestamp of claimed/ hasn't changed within 15 minutes after the 10:40 start. The exit 0 design maximizes compatibility with launchd, but in exchange carries the silent risk that "normal exit and silent failure are indistinguishable." This monitoring script isn't optional—it should be installed as a pair with the mv claim pattern.

Best Practices

Here are the rules that solidified after six months of running an autonomous environment at the ¥1.2M/month scale, each grounded in real code.

1. Put the mv ownership claim at the very top of the script

Run mv before argument validation, log initialization, or directory checks. If you can't get the file, there's no need to run any subsequent processing at all. This minimizes CPU and memory usage per lane while resolving contention as fast as possible.

TARGET=$(ls -1 "$QUEUE_DIR"/*.json 2>/dev/null | sort | head -1)
[ -z "$TARGET" ] && exit 0
BASENAME=$(basename "$TARGET")
mv "${QUEUE_DIR}/${BASENAME}" "${CLAIMED_DIR}/${BASENAME}" 2>/dev/null || exit 0
# ここより下は自プロセスだけが実行する

2. Log successful claims; keep failures (race avoidance) silent

With "success = logged, race avoidance = silent," grep "claimed:" codex-funnel.log | wc -l becomes today's processed count. A count log that needs no filtering dramatically reduces the implementation cost of periodic monitoring scripts. Pairing 2>/dev/null with exit 0 is a consistent pattern.

3. Return exit 0 only for "normal race avoidance" and "empty queue"

For "directory doesn't exist," "permission error," and "unexpected mv error," return exit 1. The more silent exits you have, the later you discover failures. Set up the environment with an initial deploy script, then write the script body on the premise that "only normal exits are exit 0," and maintenance cost goes down.

4. Make state visible with the three directories queue/, claimed/, done/

When the directory itself represents state, ls ~/dev/note-autolike/claimed/ | wc -l tells you "the number of stuck files" at a glance. No flag management or DB needed—ls alone is your monitoring tool.

5. Put epoch seconds + PID in filenames to stabilize sort order

FILENAME="job_$(date +%s)_$$_$(uuidgen | cut -d- -f1).json"

$$ is the PID of the generating process. Even when multiple files are created in the same second, differing PIDs make the lexicographic order deterministic. This stabilizes the premise of the mv claim pattern: all five lanes select the same file as first.

6. Standardize the "record the reason for skipping" pattern from reconcile.sh across all scripts

Lines 18–22 of autolike-plist-reconcile.sh log the reason and continue when a running job is found:

pid=$(launchctl list | awk -v l="$L" '$3==l{print $1}')
if [ -n "$pid" ] && [ "$pid" != "-" ]; then
  echo "[$(date '+%F %T')] $L 実行中(pid=$pid) のため見送り ($have -> $want)" >>"$LOG"
  continue
fi

The pattern of logging "why nothing was done" makes later failure tracing dramatically easier. Keep this habit even after the mv claim eliminates the race.

7. Separate post-crash recovery from the main script

If you build the "return leftover claimed/ files to queue/" step into the main script, a new race reappears where two processes simultaneously "return the leftover file → try to re-claim it." Stick to a design where recovery is delegated to an independent monitoring script that runs at low frequency.

8. Record the start time in a sidecar file right after claiming

mv "${QUEUE_DIR}/${BASENAME}" "${CLAIMED_DIR}/${BASENAME}" 2>/dev/null || exit 0
echo "{\"claimed_at\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}" \
  > "${CLAIMED_DIR}/${BASENAME}.claim"

Writing the start time to a .claim sidecar lets you write a recovery script that "returns files whose .claim is older than one hour to queue/." Aging detection for leftover files becomes precise.

9. Make the initial deploy script idempotent and re-runnable

mkdir -p ~/dev/note-autolike/{queue,claimed,done,logs}

Prepare a script that runs this before calling launchd's bootstrap. This structurally prevents a recurrence of failure #5 from part 2, where mv fails silently because a directory doesn't exist.

10. Separate log files between lanes to prevent interleaved lines

The plist settings StandardOutPath: codex-funnel.log and StandardErrorPath: codex-funnel.error.log point to separate files per lane, and that's correct. In the post-recovery situation where five lanes start at once, if a shared log gets interleaved, reconstructing what happened becomes impossible. Including the label name (com.lily.codex-note-funnel) in the log filename is the minimum bar.

11. Be aware of ThrottleInterval behavior and make the setting's intent explicit

If you consistently use exit 0, launchd's default ThrottleInterval causes no problems. But consciously choose to "leave it at the default" and write the reason in the plist or operations docs. This prevents the confusion of "I didn't change any settings, so why?" when trouble hits six months later.

12. Verify the same filesystem with df -h before placing queue/ and claimed/

The biggest risk to the atomicity premise is a cross-device mv. Build the habit of checking whenever you change directories into the deploy script:

queue_dev=$(df -P "$QUEUE_DIR" | awk 'NR==2{print $1}')
claimed_dev=$(df -P "$CLAIMED_DIR" | awk 'NR==2{print $1}')
[ "$queue_dev" = "$claimed_dev" ] || { echo "ERROR: cross-device mv"; exit 1; }

Summary

The problem that occurred on September 17, 2026, after Codex recovered from usage_limit_exceeded—"the funnel-pm lane processed the same JSON 4 times, one minute apart"—was an inevitability of the design. When five lanes fire at once with 57 JSON files backed up, a pickup script without mutual exclusion collides structurally, not probabilistically. Because launchd is a scheduler, not a mutex.

The fix was one line.

mv "$TARGET" "$CLAIMED_DIR/$BASENAME" 2>/dev/null || exit 0

POSIX rename(2) is serialized by the kernel on the same filesystem. Fundamentally unlike a "check, then acquire" PID check, the check and the acquisition complete within a single system call. Unlike flock with its fd-inheritance problem, the lock is decoupled from the life and death of the process. Since the file itself becomes the claim token, a leftover file after a crash remains in a state that's human-readable.

The "do nothing if running" PID-check pattern in autolike-plist-reconcile.sh and the mv claim pattern here were born from the same philosophy. The only difference is "whether there's a TOCTOU window." Both are designs that make the choice "exit if someone else is processing" in the very first operation.

To keep an autonomous environment stable over the long term, the only way is to keep stacking up these kinds of "structures that don't permit contention in the first place." Procedural fixes shrink the wound; environmental fixes eliminate the event. The reason I could go from ¥0 to ¥1.2M a month in six months is that on the days things broke, I repeatedly chose "fix the structure" over "next time I'll do it differently."

The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day setup guide are compiled in a paid note (Japanese).
📕 How to actually earn with a Claude Code autonomous environment — the system, real examples, getting started, and support

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

2 Pitfalls in Priority Probes: Letting One Real Request Through an Open Circuit Breaker

Lily — Sun, 20 Sep 2026 00:00:06 +0000

A circuit breaker that stops everything is easy to reason about — until the one job that can't afford to wait gets stopped along with everything else. In my setup, that job was posting: it fell to a 43% execution rate (231 runs vs. 308 skips) because engagement jobs had burned through the quota first. This post is about the fix inside claude-quota-guard.py — claim_priority_probe and run_job, which let a single real request through while the circuit is still open — and the two pitfalls I hit along the way, one of which silently dropped a daily job for four days (2026-09-13 to 16).

Last time, I wrote about how the gate blocks one row, not the whole batch. Before I get to quota-catchup.py — the script that re-runs everything once it detects quota recovery — I want to look at how the SKIPPED markers it reads are actually produced.

The problem: the circuit stops every job equally, but posting can't wait

When claude-quota-guard.py detects the quota limit, it returns EXIT_CIRCUIT_OPEN (75) until open_until and stops all 15 guarded jobs uniformly. A comment in the code explains why that wasn't enough:

# 🔴 2026-08-21: circuit が開くと全ジョブが一律で止まるため、消費の大半を占める
# 返信/エンゲージ系がクォータを使い切った巻き添えで「投稿」まで停止していた。
# 実測(launchd.log 累計): xpilot.autopost は 231実行/308スキップ＝実行率43%で、
# threadspilot.engage(64%) より優先度が低い扱いになっていた。投稿はその時間帯を逃すと
# 二度と埋まらないので、--priority を付けたジョブだけは circuit が開いていても
# この間隔で1回だけ試行を許す。試行が通ればクォータ回復の早期検知にもなる
# (従来は open_until まで盲目的に待つだけだった)。

(claude-quota-guard.py:18-24)

The measured result: stopping everything uniformly meant the posting job (xpilot.autopost) got caught in the crossfire of the quota-hungry engagement jobs and dropped to a 43% execution rate — effectively lower priority than threadspilot.engage (64%). On top of that, open_until is determined either by "the reset time parsed from the limit message" or by "a 6-hour cooldown" (record_claude_result), so even if the actual quota comes back earlier, the circuit dutifully stays open until that time. For a job like posting, where "if you miss the time slot, it never gets filled," that's not something you can ignore.

The design: give `--priority` jobs one attempt every 30 minutes

The fix is: "even while the circuit is open, let priority jobs — and only priority jobs — send one real request at a fixed interval." The caller passes --priority when handing a command to run_job.

def run_job(label: str, command: list[str], priority: bool = False) -> int:
    if not command:
        print("claude quota guard: --job requires a command after --", file=sys.stderr)
        return 2
    status = circuit_status()
    probe_claimed = False
    if status["is_open"]:
        if not (priority and claim_priority_probe()):
            print(
                "CLAUDE_QUOTA_JOB_SKIPPED "
                f"job={label} reason={status['reason']} remaining={status['remaining_seconds']}s ts={now()}",
                file=sys.stderr,
            )
            return 0
        probe_claimed = True
        print(
            "CLAUDE_QUOTA_JOB_PRIORITY_PROBE "
            f"job={label} reason={status['reason']} remaining={status['remaining_seconds']}s ts={now()}",
            file=sys.stderr,
        )

(claude-quota-guard.py:471-490)

Even with priority set, whether the job can actually attempt anything is decided by claim_priority_probe().

def claim_priority_probe() -> bool:
    """circuit が開いている間、優先ジョブに試行権を1つ渡す。

    間隔は全優先ジョブで共有する(=1本が使ったら次の枠まで他も待つ)。上限に本当に
    達している間に何本も叩いてもクォータは戻らないため、叩く回数自体を絞る。
    """
    if PRIORITY_PROBE_INTERVAL <= 0:
        return False
    with locked_state() as state:
        last = int(state.get("last_priority_probe") or 0)
        current = now()
        if current - last < PRIORITY_PROBE_INTERVAL:
            return False
        state["last_priority_probe"] = current
        return True

(claude-quota-guard.py:454-468; PRIORITY_PROBE_INTERVAL defaults to 1800 seconds at claude-quota-guard.py:25)

last_priority_probe is a single timestamp stored in locked_state() (a JSON-persisted state guarded by fcntl.flock). The key point is that this value lives per circuit, not per job: it doesn't care who probed, only "has it been 30 minutes since anyone last probed?"

The side that wins the attempt passes CLAUDE_QUOTA_PRIORITY_PROBE=1 in the environment of the child process it launches via subprocess.run.

    guard = str(Path(__file__).resolve())
    env = os.environ.copy()
    env["CLAUDE_AUTOMATION_GUARD"] = "1"
    env["CLAUDE"] = guard
    env["CLAUDE_BIN"] = guard
    if probe_claimed:
        # 内側の run_claude に「プローブとして走っている」ことを伝える(これが無いと circuit で即 75 になる)
        env["CLAUDE_QUOTA_PRIORITY_PROBE"] = "1"

(claude-quota-guard.py:491-498)

The receiver of this flag is run_claude. When the job command internally invokes the real claude binary, PATH has been rewired so the call goes through the guard itself — which means a second circuit_status() check runs inside the child process.

def run_claude(arguments: list[str]) -> int:
    status = circuit_status()
    if status["is_open"]:
        # run_job が優先プローブを claim した子プロセスだけは circuit を素通りして実 claude を叩く。
        # 結果は record_claude_result に入るので、上限文なら circuit が延び、成功なら閉じる。
        if os.environ.get("CLAUDE_QUOTA_PRIORITY_PROBE") != "1":
            print(
                "CLAUDE_QUOTA_CIRCUIT_OPEN "
                f"reason={status['reason']} remaining={status['remaining_seconds']}s",
                file=sys.stderr,
            )
            return EXIT_CIRCUIT_OPEN
        print(
            "CLAUDE_QUOTA_PRIORITY_PROBE_PASS "
            f"reason={status['reason']} remaining={status['remaining_seconds']}s",
            file=sys.stderr,
        )

(claude-quota-guard.py:416-432)

If you forget to propagate the environment variable, run_job wins the attempt, but the inner run_claude checks the circuit again and immediately returns EXIT_CIRCUIT_OPEN. The one line that gets you through both layers of the gate is env["CLAUDE_QUOTA_PRIORITY_PROBE"] = "1".

And the result of the actual call flows into record_claude_result as usual. If the limit message still comes back, open_until is extended; if the call succeeds, the circuit closes. Whether the probe fails or succeeds, that single result directly determines the circuit's next state — which is what makes this "one real request let through."

Pitfall 1: "one every 30 minutes" is a counter shared by all priority jobs

As the docstring on claim_priority_probe says, PRIORITY_PROBE_INTERVAL is shared across all jobs, not tracked per job. Even if several --priority jobs are scheduled inside the same 30-minute window, the moment the first one passes claim_priority_probe(), last_priority_probe is updated. Every subsequent job evaluates to False at priority and claim_priority_probe() and falls through to SKIPPED without ever touching the real client.

This is intentional. While the limit is genuinely in effect, hammering it with multiple requests won't bring the quota back, so the design throttles the number of attempts themselves. Operationally, though, if you forget that "having multiple priority jobs does not mean each gets its own 30-minute opportunity," you will lose time wondering "why does this one job never get a turn to verify recovery?"

Pitfall 2: when the probe fails, it silently lands as `RAN exit≠0`

This is the main subject of this post. After subprocess.run, run_job checks whether the probe came up empty and emits a different marker accordingly.

    try:
        result = subprocess.run(command, env=env, check=False)
    except OSError as exc:
        print(f"claude quota guard job={label}: {exc}", file=sys.stderr)
        return 127
    if probe_claimed and result.returncode != 0 and circuit_status()["is_open"]:
        # 優先プローブが上限のまま空振りした。RAN exit≠0 のまま残すと quota-catchup.py
        # （最新マーカー=SKIPPED だけを再実行）から漏れ、復帰後も当日分が欠番になる
        # （2026-09-13〜16 の note2-daily / codex-note-funnel 実測）。SKIPPED として記録する。
        print(
            "CLAUDE_QUOTA_JOB_SKIPPED "
            f"job={label} reason=priority-probe-quota exit={result.returncode} ts={now()}",
            file=sys.stderr,
        )
        return result.returncode
    print(
        f"CLAUDE_QUOTA_JOB_RAN job={label} exit={result.returncode} ts={now()}",
        file=sys.stderr,
    )
    return result.returncode

(claude-quota-guard.py:499-518)

Before this branch existed, the code only looked at the fact that the probe had won the attempt and actually launched a child process, and fell straight through to the trailing CLAUDE_QUOTA_JOB_RAN. Even when the probe hit the limit again and ended with exit≠0, what remained in the log was CLAUDE_QUOTA_JOB_RAN job=... exit=1.

The problem is that this RAN marker means "already done" as far as quota-catchup.py is concerned. Here is latest_job_marker, which quota-catchup.py uses to narrow down re-run candidates:

def latest_job_marker(paths: list[Path], label: str) -> Optional[Tuple[str, int]]:
    """Return the newest timestamped skip/run marker for one launchd label."""
    marker_pattern = re.compile(
        r"CLAUDE_QUOTA_JOB_(SKIPPED|RAN)\s+job="
        + re.escape(label)
        + r"(?=\s|$).*\bts=(\d+)(?=\s|$)"
    )
    ...


def latest_marker_is_today_skip(paths: list[Path], label: str, today: datetime.date) -> bool:
    marker = latest_job_marker(paths, label)
    if marker is None:
        return False
    marker_type, timestamp = marker
    return marker_type == "SKIPPED" and datetime.fromtimestamp(timestamp).astimezone().date() == today

(quota-catchup.py:160-193)

As you can see, this check only looks at the marker type (SKIPPED or RAN) and never inspects the exit code on the RAN side. Even if the probe failed with exit=1, as long as the last log line is CLAUDE_QUOTA_JOB_RAN, latest_marker_is_today_skip returns False and the job quietly drops out of find_candidates' re-run set.

The consequence: even after the circuit really closes, the job sits there with the wrong record — "already RAN today" — until its next scheduled time (the following morning, for instance). The comment records the real-world impact: note2-daily and codex-note-funnel both lost their daily run through this path between 2026-09-13 and 16.

The fix is simple. Check the condition "the probe came up empty while the circuit was open" (probe_claimed and result.returncode != 0 and circuit_status()["is_open"]) first, and only in that case emit CLAUDE_QUOTA_JOB_SKIPPED reason=priority-probe-quota instead of CLAUDE_QUOTA_JOB_RAN. The function's return value stays result.returncode, unchanged. launchd's LastExitStatus still records the actual failure correctly, while only the log marker that quota-catchup.py reads gets relabeled as "still needs a retry." That's the separation of concerns.

Note: The exit code and "should this be retried?" are separate axes. This bug happened because the two had been crammed into a single RAN marker, and the quota-catchup.py side never anticipated the exit≠0 case. When designing log markers, it's safer not to let "what actually happened" and "what the downstream batch should do next" share the same string.

Summary

The circuit stops everything uniformly, but jobs that can't recover a missed opportunity — like posting — can hold one attempt every 30 minutes via --priority
The attempt right is managed by claim_priority_probe through last_priority_probe in locked_state, and it is shared across all priority jobs, not per job. If multiple priority jobs land in the same window, there's only one opportunity
The child process that wins the attempt must receive CLAUDE_QUOTA_PRIORITY_PROBE=1. Forget it, and the inner run_claude rejects the call at the second circuit check
When the probe comes up empty, record it as CLAUDE_QUOTA_JOB_SKIPPED reason=priority-probe-quota, not CLAUDE_QUOTA_JOB_RAN exit≠0. The re-run check in quota-catchup.py only looks at the marker type and never at the exit code
Design log markers so that "what actually happened" and "what the downstream batch should do" are kept separate

Next time, I'll cover how quota-catchup.py picks up these SKIPPED markers and decides how much to re-run after recovery.

If you run a circuit breaker in front of your own scheduled jobs: does your downstream retry logic distinguish "ran and failed" from "never really got a chance"?

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

13 Days of Silence: How One Bad Draft Froze an Entire DM Lane

Lily — Sat, 19 Sep 2026 11:00:05 +0000

Last time I wrote about a Terminal-watching daemon that auto-clicks Computer Use approval dialogs. This post is the same genre—an unattended job dying quietly—but the culprit wasn't a gap in monitoring. It was the design itself. founder-scout's automated DM lane sent zero messages for 13 days because of a single bad record, and nobody noticed.

The problem: the exit code was there, but nobody saw it

founder-scout has a followers lane that sends a first DM to new followers of @bokuwalily. When I checked on 2026-09-10, I found that the last successful send was 8/28 at 16:35. Thirteen days, not a single DM.

The queue (state/followers_queue.jsonl) kept growing with every run, from 336 entries to 390. In other words, drafts were being generated daily and the send script was launching daily. Still zero sends. The logs recorded process.exit(3) day after day, but that abort path was never wired to a Discord alert, so it never crossed anyone's eyes.

Root cause: turning "one bad record" into "everything stops"

On 8/28 I wired a number-gate (checkColdOutbound) into all seven outbound paths to stop drafts from leaking my actual download counts or revenue figures. At that time I designed followers_send.mjs and dm.mjs so that if even one draft in the batch leaked a number, the script would immediately process.exit(3) and halt everything.

The leaking draft was addressed to @kouAI_work and contained the phrase "downloads on August 25…". That one record was never written to the ledger, so every run picked the same six entries from the head of the queue, and every run stopped on the same one. The sendable drafts behind it were never evaluated once—they just kept piling up in the queue.

Note
The same day, a different incident of the same shape happened on the dm.mjs side. Composer timeouts for the same three people counted as three consecutive failures and tripped the stop guard, so the four people behind them didn't get sent for two runs in a row. Different cause, but the same pattern: the head of the queue gets pinned → the job stops there every time.

Fail-closed is for stopping that one record

This is the lesson. Pre-send checks like the number-gate or composer validation exist to keep one dangerous message from going out. But if you implement them as "any anomaly → exit the whole thing," then unless that anomaly heals itself, the entire lane stops permanently. An unrecorded anomaly stays parked at the head of the queue, so it never heals itself.

The fix has two parts.

1. Record the dropped row in the ledger and treat it as exhausted

Here's the relevant section of src/followers_send.mjs.

// src/followers_send.mjs:84-97
// 送信前ゲート: 自分のDL数・売上の実数が入っている下書きは「その1件だけ」外して台帳に記録する。
// 1件で全体を止めると、その下書きがキュー先頭に残り続けてレーンごと止まる
// （実測2026-08-28〜09-10: kouAI_work 1件で13日間0通）。
{
  const gateAt = new Date().toISOString();
  const leaks = drafts
    .map((t) => ({ who: t.handle || '?', bad: checkColdOutbound(t.dm || '') }))
    .filter((x) => x.bad.length);
  for (const l of leaks) {
    console.error(`number gate: 除外 @${l.who} — ${l.bad.map((b) => b.why).join('/')}`);
    appendSync(SENT, { handle: l.who, ok: false, reason: 'number-gate: ' + l.bad.map((b) => `${b.why}(${b.hit})`).join('/'), follow: 'skipped', at: gateAt, lane: LANE });
    exhausted.add(l.who);
  }
}
const sendable = drafts.filter((d) => !exhausted.has(d.handle));
const batch = sendable.slice(0, Math.min(perRun, laneLeft, accountLeft));

process.exit(3) is gone. Instead, it writes ok:false, reason:'number-gate: …' to the SENT ledger, marks only that one entry as exhausted, and the rest proceed to the batch as usual. dm.mjs got exactly the same treatment.

// src/dm.mjs:82-94
// 送信前ゲート: 数字漏れの下書きは「その1件だけ」外して台帳に記録する。全体を止めると
// 先頭に残り続けてレーンごと止まる（followers レーンで13日間0通の実測あり）。
{
  const gateAt = new Date().toISOString();
  const leaks = queue
    .map((t) => ({ who: t.handle || '?', bad: checkColdOutbound(t.dm || '') }))
    .filter((x) => x.bad.length);
  for (const l of leaks) {
    console.error(`number gate: 除外 @${l.who} — ${l.bad.map((b) => b.why).join('/')}`);
    recordSent({ handle: l.who, ok: false, reason: 'number-gate: ' + l.bad.map((b) => `${b.why}(${b.hit})`).join('/'), follow: 'skipped', at: gateAt });
    exhausted.add(l.who);
  }
}
const sendable = queue.filter((r) => !exhausted.has(r.handle));

Because it's now in the ledger, from the next run onward this entry is excluded at the point where the exhausted set is built (followers_send.mjs:49-51 and dm.mjs:50-52, where the string number-gate is picked out of reason and added to exhausted). The moment it was recorded, this anomaly became one that "automatically disappears on the next run."

2. Move anyone with a failure history to the back of the queue

The other fix is a reorder so that the three-consecutive-failures guard (break when consecutiveFailures >= 3) doesn't keep hitting the same faces every time.

// src/followers_send.mjs:81-83
// 一度失敗した相手は後ろへ回す。先頭の同じ数人が毎run失敗すると「3連続失敗で停止」に
// 当たり、後ろの送れる人まで道連れになる（実測2026-09-10: 同じ3人で2run連続0通）。
.sort((a, b) => (attempts.get(a.handle) || 0) - (attempts.get(b.handle) || 0));

It just sorts by attempts (the number of past send attempts) in ascending order. Now people who are structurally destined to keep failing—DMs not open, composer timing out—no longer monopolize the head of the queue, and the sendable people behind them no longer get dragged down every run.

Monitoring: an exit code alone doesn't mean "it's working"

The reason this went unnoticed for 13 days is that the logs from the broken abort path only lived locally. Seeing "loaded" in launchctl list or pm2 list only means the job is executing; whether it's producing results is a separate question.

You need to check three things together: ① is the job loaded, ② what's the distribution of exit codes in recent logs, ③ what does the actual platform show (in this case, the real count of DMs sent). In this incident, ① and ② looked fine, and it was only when I checked ③ that "zero DMs in 13 days" came to light. On top of that, I made every run log laneToday (the lane's send count for the day) and added an operational rule: three consecutive days at 0 is an anomaly. The log line at followers_send.mjs:101 is what feeds that.

console.log(`queue=${sendable.length} laneToday=${laneToday}/${perDay} accountToday=${sentToday.length}/${cfg.limits.dmPerDay} thisRun=${batch.length} held=${held}`);

Pitfalls I hit

Designing process.exit(3) on a single anomaly → an unrecorded anomaly stays parked at the head of the queue and the lane stops permanently
Forgetting to wire a Discord alert to the abort path → an exit code in a log nobody reads is the same as silence
Skipping the dropped row without writing it to the ledger → the next run re-evaluates the same draft in the same position and gets stuck on the same one every time
The three-consecutive-failures guard hitting the same pinned people every run → simply moving anyone with a failure history to the back keeps the sendable people behind them from being taken down with them
Judging "running" from the loaded state in launchctl list alone → it's only healthy once you've checked the exit codes and the real counts on the platform side

Takeaways

A fail-closed gate is written to stop that one record, not to stop the lane
Halt-everything plus not-recorded equals permanent stop unless it clears on its own next run. Whenever you write a halt, ask: "Will this stop condition naturally disappear on the next run?"
The fix was a two-line prescription: write the dropped row to the ledger with ok:false, reason and mark it exhausted, and sort anyone with a failure history to the back of the queue by ascending attempts
The loaded state in launchctl list or an exit 0 isn't enough for monitoring. Only when you've looked at the distribution of exit codes plus the real counts on the platform can you say "it's working"

Do you have an unattended job right now whose "healthy" status you've only ever verified from the process list—and when did you last check the actual output on the other end?

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

3 Weeks of Silent Failure: How One Missing `--model` Flag Drained My Interactive Quota Across 4 Jobs

Lily — Sat, 19 Sep 2026 05:00:06 +0000

I went from ¥100k a month as a student to ¥600k juggling side gigs, then to ¥0 after a layoff—and six months of building an autonomous Claude Code environment brought me to ¥1.2M a month in revenue. The difference wasn't working harder. It was betting on "grow the environment" instead of "do the work."

Why This Setup Works

Look up how to make money on the side and you'll always land on the same three options: sell your skills, make content, or build a service. They're all correct. But they share one ceiling: your time is finite.

Back when I was juggling gigs at ¥600k a month, I genuinely hit the wall on working hours. Every article, every DM, every proposal needed my own hands. The more clients I had, the more quality slipped. I couldn't add more, and I couldn't let quality drop—and that's exactly when the layoff came. I decided I wasn't going to do it the same way again.

For the restart, I chose a different approach: build the environment that runs without me first. Today, launchd jobs fire at 8:00 and 10:35 every morning, generate a stock of articles, and post a notification to Discord. Whether my Mac is open or I'm asleep, the article stock keeps growing.

There are three reasons this setup is strong.

Time accumulates. Human work consumes time. Environment work produces it. Every job run stacks up as inventory that doesn't disappear. Even when I'm sick and can't do anything for three days, the jobs quietly keep growing the stock. "As long as they're running correctly," that is—and that "if" is the whole subject of this post.

The scale changes by an order of magnitude. A human can write a few articles a day at most. Jobs run in parallel, as many as you like. Right now, four lanes—article generation, daily PDCA, structuring external data, and triaging incoming DMs—all run during the same night. I passed the throughput of a single person because agents were lined up side by side.

How deeply you understand quota structure changes. Someone who uses Claude Code as a "tool to ask the AI" and someone who wires it in as "part of the infrastructure" have fundamentally different monthly throughput. To use it as infrastructure, you have to know exactly how quota is shared and how it's consumed, or you will get stuck. This outage happened precisely in that gap of knowledge.

The Overall Flow

Here's the skeleton of the content-generation pipeline up front. I'll dissect the outage in the second half, so having the big picture first will make it easier to follow.

launchd
 ~/Library/LaunchAgents/com.shun.article-daily.plist
 起動タイミング: 08:00 / 10:35（毎日）
     │
     ▼
 ~/.claude/scripts/claude-quota-guard.py
     --job com.shun.article-daily
     │  ← クォータ残量のガード層
     ▼
 ~/.discord/run-and-notify.sh
     "zenn" "Zenn記事ストック生成"
     │  ← Discord通知ラッパー
     ▼
 ~/.claude/scripts/article-daily-stock.sh  apply
     │  ← 実際の生成ロジック
     ▼
 claude -p "プロンプト..." [--model ???]
     │
     ├─ --model を明示した場合
     │       指定モデルの独立した枠で実行
     │       → 記事テキストが正常に返る ✅
     │
     └─ --model を省略した場合
             対話セッションの「既定モデル枠」を共有消費
             → 枠が切れると以下の1行だけ stdout に吐いて終了:
                 "You've reached your Fable limit."
             → exit code は非0
             → ただしログには「1行返ってきた」としか残らない
             → [ -s output ] 等の素朴なチェックを通り抜ける ❌
             → 何週間でも誰も気づかない ← 実際に3週間気づかなかった

The bottom of the diagram is the heart of the outage. claude -p does not error when you omit --model. It quietly goes off and consumes the interactive session's quota. When that quota runs out, it returns a single-line message and exits. That one line gets mistaken for "there was output," and detection slips through.

The plist Implementation

Here's an excerpt from the actual com.shun.article-daily.plist (real paths replaced with ~).

<key>ProgramArguments</key>
<array>
  <string>~/.claude/scripts/claude-quota-guard.py</string>
  <string>--job</string>
  <string>com.shun.article-daily</string>
  <string>--</string>
  <string>/bin/bash</string>
  <string>~/.discord/run-and-notify.sh</string>
  <string>zenn</string>
  <string>Zenn記事ストック生成</string>
  <string>/bin/bash</string>
  <string>~/.claude/scripts/article-daily-stock.sh</string>
  <string>apply</string>
</array>

<key>StartCalendarInterval</key>
<array>
  <dict>
    <key>Hour</key><integer>8</integer>
    <key>Minute</key><integer>0</integer>
  </dict>
  <dict>
    <key>Hour</key><integer>10</integer>
    <key>Minute</key><integer>35</integer>
  </dict>
</array>

<key>StandardOutPath</key>
<string>~/.claude/logs/article-daily.out.log</string>
<key>StandardErrorPath</key>
<string>~/.claude/logs/article-daily.err.log</string>

<key>ProcessType</key><string>Background</string>
<key>LowPriorityIO</key><true/>
<key>Nice</key><integer>10</integer>

ProcessType: Background, LowPriorityIO: true, and Nice: 10 go in as a set. By explicitly telling the OS I/O scheduler "this is a low-priority background task," they keep the job from interfering with foreground work (opening Figma, looking something up in Safari).

The call chain is four levels deep: launchd → quota-guard → run-and-notify → article-daily-stock. claude -p is invoked inside article-daily-stock.sh, and whether --model is present there is the whole question.

Four Lanes Surfaced at Once on September 3, 2026

In my case, the same omission surfaced in four jobs simultaneously. Quoting the record left in my knowledge base:

Lane	Symptom	Damage window
`gen-column.mjs` (LINE column generation)	exit 1 every morning with the one-line "Fable limit". Stock dwindled to 2 remaining	2026/8/13–9/3 (3 weeks)
`triage` (incoming DM classification)	Every quota-exhausted run fell through to "unclassifiable"; not a single incoming DM had been auto-replied since 9/2	9/2–
`brand-404` (IG daily PDCA)	Zero output due to `Fable limit`. That day's action items and daily report never generated	9/2
`hosei-grad-planner` (PDF structuring)	All 3 runs at 00:21 / 04:30 / 10:30 produced zero structured data	9/3

For three weeks, gen-column.mjs burned through its stock every morning while "looking like it was working fine." It was only discovered when the stock hit 2 remaining. The job's log recorded "1 line returned." Nobody was reading what that one line said.

"Four lanes at once" is an important fact too. If just one job breaks, it's easy to notice. When several go silent at the same time, the unease of "everything's too quiet" fades. If anything, it pulls you toward the misreading that "automation is working and my hands are free."

The root cause is a code defect: not writing --model. Combined with lax log monitoring that treats "one line = success," it went undetected for three weeks. Each hole on its own would have been easy to spot, yet stacked together they created a blind spot.

I'll dissect that structure in the second half: why omitting --model is so hard to notice, how to build one script that makes detection reliable, and how to implement a grep gate that mechanically rejects any existing plist with the flag missing.

Implementation Details

Where `--model` Goes and How to Write It

Looking at article-daily-stock.sh after the post-outage fix, there are two places where claude -p is called: topic planning (auto-generating the next topic when the queue is empty) and article body generation. Both have --model.

# キューが空のとき: 実作業からネタを自動立案（article-daily-stock.sh 327-330行目）
TOPIC_JSON=$(run_to 600 "$CLAUDE" -p "$REPLENISH_PROMPT" \
  --strict-mcp-config --mcp-config '{"mcpServers":{}}' \
  --model "${ARTICLE_MODEL:-sonnet}" --effort high \
  --output-format text --allowedTools "Read,Grep,Glob,Bash" --max-turns 20 2>>"$LOG")

# 記事本文の生成（article-daily-stock.sh 397-401行目）
run_to 1500 "$CLAUDE" -p "$PROMPT" \
  --strict-mcp-config --mcp-config '{"mcpServers":{}}' \
  --model "${ARTICLE_MODEL:-sonnet}" --effort high \
  --output-format text --allowedTools "Read,Grep,Glob,Write,Bash" --max-turns 40 >> "$LOG" 2>&1 \
  || log "WARN: claude -p exit非0（検証で弾く）"

The key is the "${ARTICLE_MODEL:-sonnet}" form. Even if the ARTICLE_MODEL environment variable is undefined, it falls back to sonnet, so the "forgot it and fell through to the default quota" accident can't happen. It also satisfies a separate operational rule: "don't hardcode model IDs in code." If you embed a model ID as a string, you end up rewriting every script when the vendor retires an alias. Make it switchable through a single variable and you can flip every lane with one line: launchctl setenv ARTICLE_MODEL haiku.

One more thing: I always include the combination --strict-mcp-config --mcp-config '{"mcpServers":{}}'. Launching from launchd doesn't go through a login shell, so the .mcp.json in the home directory gets read. If it contains external MCP server configuration, the batch may hit external networks unintentionally. Explicitly passing an empty server config completely blocks external MCP calls inside the batch.

Designing `strict_quota_message`—The 400-Character Wall

Inside claude-quota-guard.py there's a function called strict_quota_message(). This is the single most important piece of detection logic for this incident.

# claude-quota-guard.py 41-47行目
STRICT_QUOTA_PATTERNS = (
    r"(?:you(?:'ve| have) )?hit your (?:weekly |5-?hour |usage |session )?limit",
    r"claude (?:ai )?usage limit reached",
    r"approaching your (?:weekly |usage )?limit",
)
STRICT_QUOTA_MAX_CHARS = 400

# claude-quota-guard.py 159-170行目
def strict_quota_message(text: str) -> bool:
    """成功終了(exit 0)の本文が「上限メッセージそのもの」かを判定する。

    上限本文は短く、その一文だけが返る。記事本文にたまたま同じ語が出ても発火させない。
    """
    trimmed = text.strip()
    if not trimmed or len(trimmed) > STRICT_QUOTA_MAX_CHARS:
        return False
    return any(
        re.search(pattern, trimmed, flags=re.IGNORECASE)
        for pattern in STRICT_QUOTA_PATTERNS
    )

The STRICT_QUOTA_MAX_CHARS = 400 cap is the linchpin of the design. When Claude hits a limit, the string it returns is a single sentence like "You've reached your Fable limit." or "You've hit your weekly limit."—a few dozen characters at most. A properly generated article body, on the other hand, is almost never under 400 characters. This length asymmetry is what mechanically separates "this is the limit message itself" from "this is an article body that mentions limits."

The regex patterns are designed with the same thinking. The loose QUOTA_PATTERNS (used when exit is non-zero) and the strict STRICT_QUOTA_PATTERNS (used when exit is 0) are kept separate to prevent a false positive where an article whose subject happens to be "quota" trips my own circuit breaker. In fact, during the period when this check was still loose, I had an incident where "an article written about quota tripped the circuit breaker," which is how I settled on the current implementation that switches detection logic based on exit 0 vs. non-zero.

The Postcondition Gate—"There Was Output" Is Not Success

article-daily-stock.sh includes a function called article_ok() (lines 133–181). Even if claude -p returns a healthy exit code, no article is accepted unless it passes this function.

# article-daily-stock.sh 133行目
MIN_ARTICLE_BYTES=1200

# article-daily-stock.sh 173-181行目
article_ok() {
  local f="$1"
  [ -s "$f" ] || return 1
  [ "$(stat -f%z "$f" 2>/dev/null || echo 0)" -ge "$MIN_ARTICLE_BYTES" ] || return 1
  grep -qE '^title:' "$f" || return 1
  frontmatter_title_ok "$f" || return 1
  grep -qiE 'request timed out|不明な商品|TODO: *本文|\(生成失敗\)' "$f" && return 1
  return 0
}

At least 1200 bytes, a title: frontmatter key present, a title of 70 characters or fewer, and no timeout phrasing mixed in—only when all four conditions are met is the run recorded as "generation succeeded." This validation block sits immediately after claude -p, and if the file doesn't meet the bar, it's deleted and the script exits (lines 411–415). So as not to waste the day's slot, the marker file (~/.claude/logs/.article-daily-done-${TODAY}) is designed to be created "only after validation passes" (line 509).

A postcondition is evidence that "generation completed correctly." Not whether a file exists, but whether the file's contents have the expected quality, verified mechanically. Since switching to this mindset, the misreading of "the job ran = output was stocked" has gone away.

A grep Gate That Mechanically Catches the Omission

It's hard for a human to find a missing --model. The more files you have, the more visual inspection breaks down. The following one-liner sweeps across ~/.claude/scripts/ and ~/Library/LaunchAgents/ and extracts "lines that have claude -p but no --model."

# plist配下のジョブスクリプトで --model を書き忘れている箇所を抽出する
grep -rn "claude -p" ~/.claude/scripts/ \
  | grep -v -- '--model' \
  | grep -v '^Binary'

Ideally this would go into CI or a pre-commit hook, but launchd jobs aren't connected to normal CI. Instead I use the following approach. During the weekly launchctl list | grep com.shun check that confirms all jobs exist, I run the grep above alongside it. Zero lines back means everything's safe. Even one line back means something's missing somewhere.

In cases of multi-level calls—a script called from a plist calls a script which calls another—you need to follow the chain recursively down to the leaf script.

# 多段呼び出しも含めて再帰的に検索
grep -rn "claude -p\|claude --print" \
  ~/.claude/scripts/ \
  ~/dev/ \
  --include="*.sh" --include="*.mjs" --include="*.js" --include="*.py" \
  | grep -v -- '--model' \
  | grep -v '#.*claude -p'   # コメント行は除外

Without narrowing target extensions via --include, the results balloon. The four types .sh .mjs .js .py covered 90% of my environment.

Where I Got Stuck

Three Weeks Without Noticing, Until Stock Hit 2

gen-column.mjs is a job that generates one LINE column every morning. It was supposed to raise an alert once stock dropped below 5, but for some reason it never fired. When stock hit 2, I was manually looking at Discord notifications and finally thought, "something's wrong."

Digging through the logs, every morning looked like this.

[2026-08-14 08:03:12] claude -p "..." 実行
[2026-08-14 08:03:19] 出力: "You've reached your Fable limit."
[2026-08-14 08:03:19] 出力文字数: 32
[2026-08-14 08:03:19] ジョブ完了

The job "completed" and returned exit 0, so quota-guard's tally recorded it as "success." The stock-count logic was adding "normal completion = 1 article generated," so the stock count appeared to be increasing. In reality, not a single one was added.

What exposed it was counting the actual number of files in stock directly with ls. It was 2. The gap between the job's success count and the real file count was the evidence.

After that, I made two changes: stock monitoring uses the real file count instead of a counter, and a postcondition check runs after generation. A counter is a declaration of intent to "generate"; it is not evidence that generation "succeeded."

The Wording Changed and the Circuit Breaker Let It Straight Through

On September 2, 2026, it came to light that QUOTA_PATTERNS in claude-quota-guard.py did not include session limit.

That day's error message looked like this.

You've hit your session limit · resets 6:50pm (Asia/Tokyo)

This is the wording for the "5-hour session limit." It's phrased differently from the earlier "weekly limit" or "Fable limit." Because QUOTA_PATTERNS at the time didn't have this pattern, quota detection slipped through. As a result, every job that day—including 3 note lanes—kept firing blanks against the limit, and 306 lines of wasted Claude calls piled up over 4 hours.

The fix was adding r"resets?\s+(?:at\s+)?\d{1,2}(?::\d{2})?\s*[ap]m" to QUOTA_PATTERNS (claude-quota-guard.py line 37). It's a pattern that catches the combination of the verb "resets" and a time in am/pm format. That line is still in the code today, with a comment recording the live account: "2026-09-02: the 5-hour limit comes back in the form of ~."

The lesson here is the principle that quota detection must not depend on the vendor's wording. Vendors change wording. They don't announce it. Anthropic has already used multiple phrasings: "You've reached your limit," "You've hit your Fable limit," "You've hit your session limit." Pattern matching needs to be written on the "structural features of the wording," not "exact match of the wording."

The Circuit Breaker Stopped the Posting Jobs Too

On August 21, 2026, when the circuit breaker opened (= detected quota exhaustion), a problem arose where not only generation jobs but posting jobs stopped as well.

There's a record in the comments of claude-quota-guard.py.

# 🔴 2026-08-21: circuit が開くと全ジョブが一律で止まるため、消費の大半を占める
# 返信/エンゲージ系がクォータを使い切った巻き添えで「投稿」まで停止していた。
# 実測(launchd.log 累計): xpilot.autopost は 231実行/308スキップ＝実行率43%で、
# threadspilot.engage(64%) より優先度が低い扱いになっていた。投稿はその時間帯を逃すと
# 二度と埋まらないので...

Timing is everything for posting. A post that misses its 8 a.m. slot won't have the same effect if it goes up at night. Its priority is fundamentally different from a generation job's.

The cause was a design in which "the circuit breaker stops all jobs without distinction." The generation jobs were the ones that exhausted the quota, yet the posting jobs got taken down as collateral.

The remedy was adding a --priority flag (it remains in the current code as the priority: bool argument of run_job()). Priority jobs are allowed, via claim_priority_probe(), to attempt one run per PRIORITY_PROBE_INTERVAL (default 30 minutes) even while the circuit is open (claude-quota-guard.py lines 454–468). If the attempt succeeds, it's judged that "quota has recovered" and the circuit closes. With this design, one mechanism achieves two goals: "posting doesn't stop, and recovery is detected early."

Exit 0, but the Output Was a Quota Message

The hardest pattern to find was "exit code 0 with a limit message on stdout." This happens not only when --model is omitted, but also under certain conditions where claude returns the quota message as a "success."

The original quota-guard recorded exit code 0 as "success" and leaned toward closing the circuit. So when a limit message came back with exit 0, the circuit didn't open, the job was recorded as "success," and the actual output was just the one limit line—a state that could persist for cycle after cycle.

Introducing strict_quota_message() (the 400-character check described earlier) was the direct fix for this. Even with exit 0, if stdout meets that function's conditions, it's treated as quota detection and the circuit opens (claude-quota-guard.py lines 187–190).

def is_quota_response(returncode: int, text: str) -> bool:
    if returncode != 0:
        return quota_message(text)        # ゆるいパターン
    return strict_quota_message(text)     # 厳しいパターン（400字以下＋構造マッチ）

Judge by both the exit code and the output content. Since this design went in, there have been no more misses from "exit 0 false successes."

Looking back now, all four failures grew from the same single root: the property that "claude -p doesn't error even with incomplete arguments." Incomplete arguments, unexpected behavior—Claude returns something regardless. Because that "something" gets recorded as output, everything looks normal if monitoring is lax.

Rather than designing jobs on the assumption of perfection, assume imperfect execution and layer detection and recovery on top—since moving to that philosophy, accidents like a four-lane simultaneous stall haven't happened.

Pitfalls

Listed in the order I actually hit them. For every one, I thought "I'll notice eventually"—and didn't.

Omitting --model doesn't make the command error
claude -p runs silently even with incomplete arguments. When omitted, it shares the interactive session's quota, so the problem surfaces "when the interactive quota runs out," not the moment you write the batch. It doesn't break the day you forget it; it dies quietly weeks later at some unrelated moment. The farther apart cause and effect are, the harder it is to spot the cause.

A limit message comes back with exit 0
When --model is omitted and the interactive quota runs out, claude -p writes "You've reached your Fable limit." to stdout and exits. There are cases where the exit code is 0 here. A script that judges success by $? can't distinguish the limit message from normal output. [ -s output.txt ] passes too, as long as the file isn't empty. Even 32 characters counts as "there was output."

The stock counter and the real file count drift apart
In the gen-column.mjs case, every time the job "completed" with exit 0 the success counter went up, and stock "appeared to be increasing." In reality, not one was added. A counter is a declaration that "I tried to generate," not evidence that "I generated." The reason it took three weeks to notice, until stock was actually at 2, was that the monitoring metric was the counter.

Four lanes stopping at once gets misread as "automation is working"
If one job stops, it's easy to notice "something's off." When four go silent at once, the unease of "everything's too quiet" fades. If anything, it pulls you toward "my hands are free = automation is working." The four-lane stall that surfaced on 9/3 was missed for three weeks by exactly this mechanism.

Vendors change quota wording without notice
The 9/2 error came in the form "You've hit your session limit · resets 6:50pm (Asia/Tokyo)," phrased differently from the earlier "weekly limit" and "Fable limit." QUOTA_PATTERNS at the time didn't include this pattern, and it slipped past quota detection. The result was 306 lines of wasted Claude calls over 4 hours. Write detection as "exact match of the wording" and it's neutralized the instant the vendor changes the phrasing.

The circuit breaker stops posting jobs uniformly
This was the 8/21 incident. When generation jobs exhausted the quota, posting jobs stopped as collateral. Measured, xpilot.autopost's execution rate had dropped to 43%. Generation jobs can retry if they fail. A post that misses its window can never be filled in. Stopping jobs with fundamentally different priorities under the same circuit breaker was the design flaw.

launchd doesn't go through a login shell
A script started from a plist runs in a different environment than when you run it manually from the terminal. The PATH in ~/.zshrc isn't read. node and claude commands managed by nvm can't be found. That's why com.shun.article-daily.plist explicitly sets PATH under EnvironmentVariables. This is the cause of the vast majority of "works locally, dies under launchd."

.mcp.json gets read and the batch hits external networks
Under launchd, the .mcp.json in the home directory is read automatically. If there's MCP server configuration for local development, the batch goes out to connect externally without meaning to. Without --strict-mcp-config --mcp-config '{"mcpServers":{}}', a batch can do things you can't reproduce locally.

Hardcoding model IDs in scripts causes mass breakage
When the vendor retires or renames a model alias, you end up rewriting every script with the string embedded. Write it with the version baked in, like --model claude-sonnet-4-5, and every model update means a tour through all your scripts.

The "alert when stock drops below 5" never fired
Because the alert logic pulled the stock count from the counter, the alert didn't fire while the counter was "increasing" (it actually wasn't). The monitoring metric has to be the real file count on the filesystem, not the generation job's success count.

Best Practices

Distilled from the four-lane simultaneous stall and three weeks of unnoticed loss, these are the rules I still apply to every job.

1. Always specify --model on claude -p
This is the starting point for everything. Omission is forbidden. Before writing a job, make it a rule that "a claude -p without --model does not exist."

2. Write it with an environment-variable fallback

--model "${ARTICLE_MODEL:-sonnet}"

Don't write model IDs directly into scripts. The :-sonnet fallback for the undefined case prevents "forgot it and fell to the default quota," and one line—launchctl setenv ARTICLE_MODEL haiku—switches every job.

3. Put a postcondition gate immediately after claude -p

MIN_ARTICLE_BYTES=1200
article_ok() {
  [ -s "$f" ] || return 1
  [ "$(stat -f%z "$f")" -ge "$MIN_ARTICLE_BYTES" ] || return 1
  grep -qE '^title:' "$f" || return 1
  grep -qiE 'request timed out|TODO: *本文' "$f" && return 1
  return 0
}

"There was output" is not success. Minimum byte count, presence of frontmatter, no timeout phrasing mixed in—only when all three conditions are met is it recorded as "generation succeeded."

4. Monitor stock by real file count
A counter is intent; the real file count is evidence. Base the alert metric on the actual number from ls ~/.claude/stock/ | wc -l. If the counter goes up but the files don't, it's a failure.

5. Write quota detection on the "structure of the wording"
Instead of an exact match like "Fable limit", use structural patterns like these.

r"(?:you(?:'ve| have) )?hit your (?:weekly |5-?hour |usage |session )?limit"
r"resets?\s+(?:at\s+)?\d{1,2}(?::\d{2})?\s*[ap]m"

Even if the vendor changes the wording, any phrasing with the structure of "you've hit a limit" gets caught.

6. Judge by both exit code and output content

def is_quota_response(returncode: int, text: str) -> bool:
    if returncode != 0:
        return quota_message(text)        # ゆるいパターン
    return strict_quota_message(text)     # 厳しいパターン（400字以下＋構造マッチ）

To prevent exit 0 false successes, check the exit code and stdout independently.

7. Use a 400-character cap to separate quota messages from article bodies
The message Claude returns at a limit is a few dozen characters. A normal article body is almost never 400 characters or less. If len(trimmed) > 400, don't treat it as a limit message—this one line prevents the false positive where "an article about quota trips my own circuit breaker."

8. Put --strict-mcp-config --mcp-config '{"mcpServers":{}}' into batches as a pair
Under launchd, .mcp.json gets read. Explicitly passing an empty server config shuts down the accident of a batch unintentionally calling external MCP. These two flags are one setting together. One alone doesn't work.

9. Change circuit-breaker behavior by job priority
Don't stop posting jobs and generation jobs with the same circuit breaker. Mark posts with priority: true and allow one attempt per PRIORITY_PROBE_INTERVAL (every 30 minutes) even while the circuit is open. A successful attempt detects quota recovery and closes the circuit—one mechanism achieves both keeping posts alive and detecting recovery early.

10. Run the grep gate the moment you write a plist

grep -rn "claude -p\|claude --print" \
  ~/.claude/scripts/ ~/dev/ \
  --include="*.sh" --include="*.mjs" --include="*.js" --include="*.py" \
  | grep -v -- '--model' \
  | grep -v '#.*claude -p'

Zero lines means every job is safe. Even one line back means something's missing. Always run it when adding a plist. Pairing it with the weekly launchctl list check turns it into a habit.

11. The three-piece set: ProcessType: Background / LowPriorityIO: true / Nice: 10
Put this combination into every batch job. By explicitly telling the OS "low-priority background task," it won't interfere with foreground work like having Figma open or researching in a browser. The plist default sets nothing (Standard priority), so if you don't specify, your foreground gets squeezed.

12. Set the launchd PATH explicitly via EnvironmentVariables
node and claude managed by nvm exist only in the login shell's PATH. Since launchd launches don't go through a login shell, specify a PATH containing nvm's binary path under the EnvironmentVariables key. In the com.shun.article-daily.plist implementation, ~/.nvm/versions/node/v24.13.0/bin goes at the front of PATH.

13. For multi-level calls, grep recursively down to the leaf
In the chain plist → quota-guard.py → run-and-notify.sh → article-daily-stock.sh → claude -p, applying the grep gate only to the plist never inspects the leaf script. Specify .sh .mjs .js .py via --include and search recursively to the end of the call chain.

14. Leave dated incident comments in the script
Line 37 of claude-quota-guard.py still has the comment # 2026-09-02: 5時間上限は〜の形式で返る. Writing down why a pattern was added and when means that six months later, you (or Claude Code) don't have to guess "why is this pattern here" when reading the code. Incident history gets read more when it sits next to the code than in a commit message.

Summary

Omitting --model is a one-line oversight. It produced an outage that lasted three weeks, hit four lanes simultaneously, and drained stock down to 2. The command doesn't error, it returns exit 0, and the log says "there was 1 line"—when those three overlap, a human eye won't catch it for weeks.

To use automation as infrastructure, you can't design jobs on the premise that they "work perfectly." The right philosophy is to assume imperfect execution and layer detection and recovery on top. Explicit --model is the starting point; the postcondition gate, real-file-count monitoring, structural pattern detection, and the priority-aware circuit breaker are its extensions.

Today, accidents where four lanes go silent at once no longer happen. The same principles apply as jobs grow, so even with 171 jobs running, the added mental cost is close to zero. "The environment works in my place" only holds because there's a mechanism in place for it to notice when it breaks and recover on its own.

Have you ever had a scheduled job that "succeeded" every single day while producing nothing—and what was the check that finally caught it?

The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day procedure are compiled in a paid note.
📕 How to actually make money with an autonomous Claude Code environment — system, real examples, getting started, and support

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

5 Pitfalls I Hit Building a Daemon That Auto-Approves Claude Code's Computer Use Dialog

Lily — Sat, 19 Sep 2026 00:00:05 +0000

Last time I wrote about the morning my Wi-Fi was dead for five hours. This post is in the same "stop babysitting an unattended machine" vein: a small daemon that presses Enter on Claude Code's Computer Use approval dialog for you. My first attempt, launched from launchd, produced nothing but nine osascript timeout lines at 22-second intervals and never sent a single keystroke. Once I moved it to run as a child of Terminal, it went from zero approvals to actually delivering \r to the dialog within seconds.

The problem: pressing "Allow for this session" by hand, every time

When you use Computer Use (screen control) in Claude Code, a confirmation dialog appears: Computer Use wants to control these apps. Selecting "Allow for this session" and pressing Enter gets you through, but it comes back every single time you cross a session boundary. I wanted to know whether it could be made permanent via settings, so I ran strings on the binary to check.

computerUseMcpState.allowedApps（セッション内メモリ）にしか積まれず初期値は空
settings.json / ~/.claude.json に事前付与キーは無い
dialog kind=computer_use_approval（requestDialog系）で PermissionRequest hookも通らない
bypassPermissionsModeAccepted: true でも出る

That's my field log from tracing the claude 2.1.266 binary with strings. The approval state lives only in session memory, in computerUseMcpState.allowedApps, and vanishes the moment you cross a process boundary. The same investigation did turn up settings that can be made permanent: bypassPermissionsModeAccepted: true in ~/.claude.json and projects[*].hasTrustDialogAccepted: true (I set 85 of them in one pass) both work, but they only suppress the "dangerous operation" confirmation, which is a different thing from the Computer Use approval dialog. In other words, there's no room to fix this on the settings side. That was the conclusion strings nailed down.

Note: If you put a * in the middle of an allow rule string (for example grep's [^}]* regex), you get a "wildcard before the rest" warning and the rule is rejected. Putting the wildcard at the end is the safe option.

If settings won't do it, the only option left is to look at the screen and press Enter on the human's behalf. So I wrote ~/.claude/scripts/cu_dialog_autoallow.py.

Implementation: poll every Terminal tab, and don't misfire

What it does is simple: poll every tab in Apple Terminal every 2 seconds, and when the dialog string shows up, send \r. But "see it, press Enter immediately" causes accidents. The implementation leans heavily toward misfire prevention.

READ = f'''
tell application "Terminal"
  set out to ""
  repeat with w in windows
    set k to count of tabs of w
    repeat with i from 1 to k
      set c to contents of tab i of w
      set out to out & (tty of tab i of w) & "{SEP}" & c & "{SEP}"
    end repeat
  end repeat
  return out
end tell
'''

The history property is heavy, around 600KB per tab, so I read only the visible screen (contents of tab i of w). There's also a trap: if you iterate with repeat with t in tabs and try to grab contents of t, you get the tab object itself back. You have to index explicitly in the form tab i of w.

The core of misfire prevention is checking which option the cursor is pointing at before firing.

def selected_is_allow(text):
    # the highlighted option line starts with the pointer glyph
    tail = text.split("Enter to confirm")[0]
    lines = [l.strip() for l in tail.splitlines() if l.strip()]
    for l in reversed(lines[-6:]):
        if l.startswith("❯") or l.startswith(">"):
            return "Allow for this session" in l
    return False

The line starting with ❯ (or > for ASCII-only environments) is the pointer line, and the only thing checked is whether it contains "Allow for this session". If the dialog's default cursor is on the Deny side and you send Enter anyway, you get a denial, so skipping this step makes things actively worse.

if "Computer Use wants to control these apps" in text and "Enter to confirm" in text:
    now = time.time()
    if now - last_sent.get(tty, 0) < 8:
        continue
    if not selected_is_allow(text):
        log(f"{tty}: dialog visible but cursor not on Allow; skipping")
        last_sent[tty] = now
        continue
    res = press_enter(tty)

The way Enter is sent is also designed so it doesn't steal focus.

def press_enter(tty):
    scpt = f'''
tell application "Terminal"
  repeat with w in windows
    repeat with t in tabs of w
      if tty of t is "{tty}" then
        do script "" in t
        return "sent"
      end if
    end repeat
  end repeat
  return "notfound"
end tell'''

do script "" in t just writes a single newline byte to that tab's tty; it doesn't activate the window. Sends are deduplicated per tty with an 8-second window, and fcntl.flock prevents multiple instances from running.

The trap I fell into: launching from launchd hangs silently

My first attempt was to launch it from a recurring launchd job. The result: AppleEvents from python3 to Terminal hang silently under TCC (Automation). No prompt, nothing. It just freezes. Here's the actual log.

2026-09-12 15:32:35 osascript timeout
2026-09-12 15:32:57 osascript timeout
2026-09-12 15:33:19 osascript timeout
2026-09-12 15:33:41 osascript timeout
2026-09-12 15:34:03 osascript timeout
2026-09-12 15:34:25 osascript timeout
2026-09-12 15:34:47 osascript timeout
2026-09-12 15:35:09 osascript timeout
2026-09-12 15:35:31 osascript timeout

You can see osascript timeout repeating endlessly at 22-second intervals. A process launched from launchd sits outside the user's TCC context, so its control request to Terminal can't even surface an approval dialog and gets swallowed whole.

Note: TCC (Automation) is nastier than "a permission dialog appears and gets denied": it hangs silently without ever showing a prompt. If you're sending AppleEvents via launchd, suspecting this failure mode is the fastest route to a fix.

The workaround: launch as a child process of Terminal

The workaround was to launch it as a child process of Terminal itself. AppleEvents from a child process of the same app don't trigger a TCC permission request at all. Here's what's at the end of my ~/.zshrc.

# Claude Code computer-use「Allow for this session」を自動Enter（Terminal起点で起動=TCC不要・launchd起点はTCCで固まる）
if [[ "$TERM_PROGRAM" == "Apple_Terminal" ]] && ! pgrep -qf cu_dialog_autoallow.py; then
  (nohup /usr/bin/python3 ~/.claude/scripts/cu_dialog_autoallow.py >/dev/null 2>&1 &)
fi

TERM_PROGRAM restricts this to shells started from Apple Terminal, pgrep stops a second instance, and nohup keeps it resident in the background. This block runs every time you open a new tab, but thanks to the pgrep guard only one process ever actually starts. After the switch, start pid=... shows up in the log and \r is actually being delivered to the dialog.

2026-09-12 15:36:16 /dev/ttys010: CU dialog -> Enter (sent)
2026-09-12 15:36:18 /dev/ttys011: CU dialog -> Enter (sent)
2026-09-12 15:36:26 /dev/ttys010: CU dialog -> Enter (sent)

Pitfalls I hit

Launched from launchd, AppleEvents hang silently under TCC → launch as a child of Terminal via nohup at the end of .zshrc
The history property is heavy at 600KB per tab → read only the visible screen with contents of tab i of w
repeat with t in tabs; contents of t returns the tab itself → use explicit index access with tab i of w
Sending Enter without checking the cursor position misfires (you press it on the Deny side too) → parse the ❯ pointer line and send only when Allow is confirmed
Transient AppleScript connection drops and syntax errors show up in the log (実行エラー: 接続が無効です (-609) / syntax error ... (-2741)) → design the loop to ignore one-off timeouts and keep polling (a lost cycle is retried 2 seconds later)

That last point is backed by real logs. On the morning of 09-13 and around midday on 09-16, connection drops and syntax errors came in bursts, but every one of them recovered on its own in the restart cycle a few minutes later, and an actual dialog approval (CU dialog -> Enter (sent)) was also recorded at 23:30 on 09-16. One caveat: I haven't yet measured \r reaching a real dialog at the moment another session is holding the CU lock; the end-to-end confirmation so far is against a fake dialog screen.

Summary

Per the strings investigation, there is no settings-based route to make Computer Use approval permanent (session memory only; the PermissionRequest hook doesn't fire either)
The workaround is a design that polls the visible screen and checks the cursor position to press Enter on your behalf without misfiring
Launching from launchd hangs silently under TCC. Running as a child process of Terminal is the only workaround
One-off connection drops and syntax errors are swallowed by the polling loop and self-heal

Next time I'll write about a sibling of this AppleScript watcher daemon: tiling Terminal windows in one shot.

Have you found any other way to get past a TCC-gated AppleEvent from a background process without going through the app itself?

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

4 Automation Chrome Processes Were Killing My Real Browser: Bundle-ID Isolation and a 6-Hour Self-Repair Loop

Lily — Fri, 18 Sep 2026 11:00:06 +0000

At 13:31 JST on September 1, 2026, open -a "Google Chrome" returned error code -600 and nothing happened. Three headless Chrome processes and one headful one were already running on my Mac — all spawned by the same automation that had grown from ¥0 to ¥1.2M a month in six months. That afternoon, the whole setup got redesigned from the ground up.

Why This Setup Works

Once a solo developer's automation grows past a certain size, there's a problem you will inevitably hit: "the jobs are running fine, but at some point my own working environment broke."

On the morning of September 1, 2026, Lily's Mac had three headless Chrome processes and one headful one — four in total — running at the same time. Auto-liking on social media, thumbnail generation, per-account profile management: each was an independent job calling Playwright. And every one of them was directly exec-ing /Applications/Google Chrome.app/Contents/MacOS/Google Chrome. That was all it took.

macOS LaunchServices treats apps with the same CFBundleIdentifier as a single instance of the same app. Google Chrome's bundle ID is com.google.Chrome. If an automation job has already started a process with that bundle ID, then when you run open -a "Google Chrome" from the Dock or Spotlight, the OS decides "that app is already running, so I'll just bring the existing frontmost window forward." But the automation process usually has no window. The Chrome window a human expects never comes back. That is what -600 (procNotFound) really is.

The nasty part is that you can't notice this until it happens. The automation jobs are running normally. Exit code is 0. Logs are clean. Only when a human tries to use Chrome does nothing happen.

In general, this problem has the property that "the more automation jobs you add, the higher the probability." With one job, Chrome is held for a short time. As you go to three, five, ten jobs, Chrome being permanently held by one job or another becomes the normal state. The more automation you stack on to grow revenue, the longer your own Chrome is unusable. The causality is inverted.

There were two possible directions for a fix. One: make every automation job kill Chrome when it's done. Two: separate the automation browser from the human's Chrome at the level of its name. The former requires managing the shutdown timing of every job, and each new job creates a new gap. The latter, once built, eliminates the interference at the OS level. Only the latter is a permanent fix.

Create a separate bundle named /Applications/Chrome Automation.app and change its bundle ID to com.google.ChromeAutomation. From the OS's perspective it's a completely different app. LaunchServices manages com.google.Chrome and com.google.ChromeAutomation separately. No matter how many Chrome Automation instances the automation launches, a human's open -a "Google Chrome" looks for a different bundle ID and is unaffected.

If I had to explain in one line why this works: "Resource contention disappears the moment you separate the names." As long as they share a name, you have a structure where killing one kills the other. Split the name, and the OS isolates them for you.

There is, however, one awkward implementation wall.

Playwright's channel: 'chrome' option writes the path of the browser it launches into an internal file, playwright-core/lib/coreBundle.js. This file lives inside node_modules. Every time npm install runs, it gets overwritten and reverts to /Applications/Google Chrome.app.

In other words, even if you rewrite the path, it periodically reverts. Designing around the fact that "it reverts" is the key to stable operation. "Anticipate the revert and re-apply periodically." That's the reason the 6-hour self-repair script exists.

The Overall Flow

Here's the system architecture as a diagram.

  ┌──────────────────────────────────────────────────────────┐
  │  npm install（いつ走るかわからない）                          │
  │     ↓                                                     │
  │  playwright-core/lib/coreBundle.js                        │
  │     "...Google Chrome.app/Contents/MacOS/Google Chrome"  │ ← 元に戻る
  └──────────────────────────────────────────────────────────┘
                    ↑最大6時間以内に検知
  ┌──────────────────────────────────────────────────────────┐
  │  launchd  com.shun.chrome-automation-repair              │
  │  StartInterval: 21600（6時間ごと）RunAtLoad: true          │
  │     ↓                                                     │
  │  ~/.claude/scripts/chrome-automation-repair.sh           │
  │                                                           │
  │  1. keychain_gate() ─ --use-mock-keychain 未設定を警告      │
  │                                                           │
  │  2. バージョン比較                                           │
  │     src_ver（Google Chrome.app）                           │
  │     dst_ver（Chrome Automation.app）                      │
  │        一致 → スキップ                                      │
  │        不一致 or 不在 → rebuild                            │
  │            cp -R /Applications/Google Chrome.app         │
  │                   /Applications/Chrome Automation.app    │
  │            PlistBuddy: CFBundleIdentifier                │
  │                        → com.google.ChromeAutomation     │
  │            PlistBuddy: CFBundleName                      │
  │                        → Chrome Automation               │
  │            codesign --force --deep --sign -              │ ← ad-hoc 再署名
  │                                                           │
  │  3. playwright-core を glob で検索（find より速い）           │
  │     ~/dev/*/node_modules/playwright-core/lib/coreBundle.js│
  │     ~/content/*/node_modules/playwright-core/...         │
  │     ※ネスト4階層まで対応                                     │
  │        sed: Google Chrome.app のパス → AUTO_BIN に置換      │
  │        patched=N  already=M                               │
  │        patched+already==0 → exit 1（無音成功を防ぐ）          │
  │                                                           │
  │  ログ: ~/.claude/logs/chrome-automation-repair.log        │
  └──────────────────────────────────────────────────────────┘
                    ↓ パッチ済み
  ┌──────────────────────────────────────────────────────────┐
  │  playwright channel:'chrome' の解決先                      │
  │  /Applications/Chrome Automation.app/                    │
  │              Contents/MacOS/Google Chrome                │ ← 自動化バンドル
  └──────────────────────────────────────────────────────────┘
         ↕ 完全に独立
  ┌──────────────────────────────────────────────────────────┐
  │  open -a "Google Chrome"                                 │
  │  com.google.Chrome ← 人間の Chrome（干渉なし）               │
  └──────────────────────────────────────────────────────────┘

Let's look at the core parts of the script in real code.

Bundle Creation and Version Management

SRC="/Applications/Google Chrome.app"
DST="/Applications/Chrome Automation.app"
BUNDLE_ID="com.google.ChromeAutomation"
AUTO_BIN="$DST/Contents/MacOS/Google Chrome"

ver() { /usr/libexec/PlistBuddy -c "Print :CFBundleShortVersionString" \
        "$1/Contents/Info.plist" 2>/dev/null; }

src_ver="$(ver "$SRC")"
dst_ver="$(ver "$DST")"

if [ ! -d "$DST" ] || [ "$src_ver" != "$dst_ver" ]; then
  cp -R "$SRC" "$DST"
  /usr/libexec/PlistBuddy -c "Set :CFBundleIdentifier $BUNDLE_ID" \
      "$DST/Contents/Info.plist"
  /usr/libexec/PlistBuddy -c "Set :CFBundleName Chrome Automation" \
      "$DST/Contents/Info.plist"
  codesign --force --deep --sign - "$DST"
fi

The ver() function reads CFBundleShortVersionString (e.g. 127.0.6533.120) via PlistBuddy. When Google Chrome auto-updates, src_ver and dst_ver drift apart, so the next 6-hour cycle automatically rebuilds Chrome Automation.app.

The - (minus) in codesign --force --deep --sign - means an ad-hoc signature. The moment you rewrite the bundle ID in Info.plist, Google's signature becomes invalid. If you try to launch without re-signing, the helper processes die instantly with SIGKILL (exit 137). An ad-hoc signature is a self-signature with no trust chain, but for local-only execution that's fine.

Glob Search and Patching of playwright-core

SCAN_ROOTS=("${HOME}/dev" "${HOME}/content")

list_core_bundles() {
  local root sub
  shopt -s nullglob
  for root in "${SCAN_ROOTS[@]}"; do
    for sub in \
      "$root"/*/node_modules/playwright-core/lib/coreBundle.js \
      "$root"/*/node_modules/playwright/node_modules/playwright-core/lib/coreBundle.js \
      "$root"/*/*/node_modules/playwright-core/lib/coreBundle.js \
      "$root"/*/*/node_modules/playwright/node_modules/playwright-core/lib/coreBundle.js
    do
      [ -f "$sub" ] && echo "$sub"
    done
  done
  shopt -u nullglob
}

This sweeps the repositories under ~/dev and ~/content with glob patterns up to four levels deep. Walking the entire tree with find . -name "coreBundle.js" takes 4 minutes; hitting the layouts that actually exist (the two patterns */node_modules/playwright-core/... and */node_modules/playwright/node_modules/playwright-core/...) directly with globs brings the runtime down to 0.95 seconds.

The patch step is an in-place substitution with sed -i '':

patched=0; already=0
while IFS= read -r f; do
  if /usr/bin/grep -q "$SRC/Contents/MacOS/Google Chrome" "$f" 2>/dev/null; then
    /usr/bin/sed -i '' \
      "s|$SRC/Contents/MacOS/Google Chrome|$AUTO_BIN|g" "$f" \
      && patched=$((patched+1))
  elif /usr/bin/grep -q "$AUTO_BIN" "$f" 2>/dev/null; then
    already=$((already+1))
  fi
done < <(list_core_bundles)

if [ $((patched + already)) -eq 0 ]; then
  log "ERROR: playwright-core が1件も見つからない。SCAN_ROOTS を確認せよ。"
  exit 1
fi

patched is the number of files rewritten this run; already is the number already patched. If both are 0, the search path is broken and nothing was found. To prevent "zero results but treated as success," it explicitly fails with exit 1. This fail-loud design prevents the later question, "why did it go six hours unfixed without anyone noticing?"

Scheduled Execution via launchd

<key>StartInterval</key><integer>21600</integer>
<key>RunAtLoad</key><true/>
<key>LowPriorityIO</key><true/>
<key>Nice</key><integer>10</integer>
<key>ProcessType</key><string>Background</string>
<key>StandardOutPath</key>
  <string>~/.claude/logs/chrome-automation-repair.log</string>

StartInterval: 21600 is 21600 seconds = 6 hours. With RunAtLoad: true, it also runs once the moment you launchctl load it. Nice: 10 and LowPriorityIO: true lower CPU and disk priority so it doesn't interfere with human work in the background.

The meaning of the 6-hour interval is an SLA: "auto-repair within at most 6 hours after an npm install runs." In practice the automation jobs rarely run npm install, so in most cases the patch doesn't revert before the next cycle. Even if it does, it's fixed within 6 hours. This design accepts the "length of the repair window" as a known risk.

Keychain Gate Detection

At the top of the script there's a detection routine called keychain_gate():

keychain_gate() {
  local hits
  hits=$(grep -rIl --include='*.sh' --include='*.py' --include='*.js' \
      --exclude-dir=node_modules ... \
      -E 'Chrome Automation\.app/Contents/MacOS|Google Chrome\.app/Contents/MacOS' \
      "${SCAN_ROOTS[@]}" ~/.claude/scripts 2>/dev/null \
    | xargs -I{} grep -l -e '--headless' {} 2>/dev/null \
    | xargs -I{} grep -L -e 'use-mock-keychain' {} 2>/dev/null \
    | grep -v 'scent-media/scripts/ensure_chrome.sh' || true)
  if [[ -n "$hits" ]]; then
    log "WARNING: --use-mock-keychain 無しでChromeをheadless起動している..."
    echo "$hits" | sed 's/^/  /'
  fi
}

Because Chrome Automation.app is ad-hoc signed, even if you grant "Always Allow" on the "Chrome Safe Storage" ACL in macOS Keychain, the code hash changes every time a Chrome update triggers a rebuild, and the permission is invalidated. Any script that launches headless without --use-mock-keychain --password-store=basic causes a Keychain access dialog to pop up every time. Since this leads to "silent failures" in automation, there's a gate that detects and warns — but does not fix.

In this round of work, the repositories fixed came to 16 locations where Playwright's channel:'chrome' resolves, 10 files with direct shell and Python invocations, and 8 repositories committed. The larger the scale, the less "I fixed one place" is the end of it. I needed to propagate the fix across every repository, and have a mechanism that automatically restores the fix even when npm install wipes it out. That's why this setup exists.

Implementation Details

The pgrep Guard — Don't Rebuild While Running

The bundle-creation logic shown earlier has one more safety valve. When a version mismatch is detected, if Chrome Automation.app is running, skip the rebuild.

if pgrep -f "Chrome Automation.app/Contents/MacOS" >/dev/null 2>&1; then
  log "SKIP rebuild: 自動化Chromeが稼働中 (src=$src_ver dst=${dst_ver:-none})"
else
  log "rebuild: src=$src_ver dst=${dst_ver:-none}"
  rm -rf "$DST" || { log "ERROR: 旧バンドル削除に失敗"; exit 1; }
  cp -R "$SRC" "$DST" || { log "ERROR: コピー失敗"; exit 1; }
  ...
fi

cp -R takes 15–30 seconds. If Chrome Automation.app's Contents/MacOS/Google Chrome is read mid-copy, a half-written binary gets executed and the whole job breaks. pgrep -f "Chrome Automation.app/Contents/MacOS" looks for running processes, and if any exist, the design is to "wait until the next 6-hour cycle."

Even when the rebuild is skipped, the patched/already counting still runs afterward. The bundle itself may be stale, but as long as the path in coreBundle.js is correct, jobs keep working.

launchd's Shell Doesn't Have a Human's PATH

The plist contains an EnvironmentVariables block.

<key>EnvironmentVariables</key>
<dict>
  <key>HOME</key><string>~</string>
  <key>LANG</key><string>en_US.UTF-8</string>
  <key>PATH</key>
    <string>~/.local/bin:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
</dict>

The shell launchd starts only inherits its environment from /etc/launchd.conf; it reads neither ~/.zshrc nor ~/.zprofile. Unless you set PATH explicitly, only /usr/bin:/bin is available, and codesign and PlistBuddy in /opt/homebrew/bin can't be found. HOME is set explicitly so that the ${HOME}/dev expansion is resolved via EnvironmentVariables rather than as a shell variable. LANG is insurance so grep doesn't mangle filenames containing Japanese.

The log destinations being split into StandardOutPath and StandardErrorPath is for the same reason. launchd writes stdout and stderr to separate files, both landing in ~/.claude/logs/. If you only look at one, it can appear as though "nothing is happening."

Reading the keychain_gate Pipeline

The keychain_gate() implementation has an unusual structure: three chained greps.

grep -rIl ... \
  -E 'Chrome Automation\.app/Contents/MacOS|Google Chrome\.app/Contents/MacOS' \
  "${SCAN_ROOTS[@]}" "${HOME}/.claude/scripts" \
| xargs -I{} grep -l -e '--headless' {} \
| xargs -I{} grep -L -e 'use-mock-keychain' {}

Stage 1 finds "files that directly write the Chrome binary path." node_modules, .git, profiles, logs, venv, and tests are excluded from the search.

Stage 2 keeps only files containing --headless. Scripts that launch with a GUI are fine even if the Keychain dialog appears.

Stage 3 keeps only files that do not contain use-mock-keychain (-L is the "list non-matching files" flag).

Files that pass all three stages are in the state of "using Chrome headless but without mock-keychain." Each time they run, the Keychain "Allow?" dialog pops up. When a dialog appears mid-automation, every process stalls until the next click. The one file excluded via grep -v, scent-media/scripts/ensure_chrome.sh, is deliberately designed to use the real Keychain for a different reason.

The script does not auto-fix on this detection. Persistent profiles like Instagram encrypt their session cookies with a Keychain key, and if the key changes, the login is wiped out. Log a warning and leave the decision to a human — that's the design choice.

Final Verification Commands

Once the setup is in place, confirm it actually works.

# 1. launchd の登録確認
launchctl list | grep chrome-automation-repair

# 2. バンドルが生きているか
open -a "Chrome Automation" --args --version

# 3. playwright が正しいパスを見ているか
grep -r "Chrome Automation" ~/dev/*/node_modules/playwright-core/lib/coreBundle.js 2>/dev/null | head -5

# 4. 人間の Chrome は独立して起動できるか
open -a "Google Chrome"

Number 2's --args --version passes the --version flag to Chrome, making it print the version string at startup (e.g. Google Chrome 127.0.6533.120). If you get the string back without a window opening, the bundle is in a launchable state. Number 4 is the final confirmation: if the human's Chrome opens without returning -600, the isolation is working.

Where I Got Stuck

`find` Was Taking 4 Minutes

The first script I wrote searched with find.

find "${HOME}/dev" "${HOME}/content" \
  -name "coreBundle.js" \
  -path "*/playwright-core/lib/*"

When I ran it, it took 4 minutes 15 seconds to complete. There are several hundred node_modules directories under ~/dev, and find walks every one of them. A 4-minute process every 6 hours wastes CPU and disk even in the background, and worse, I hadn't noticed the "it's slow when you actually run it" fact until the first implementation.

The layouts where coreBundle.js actually exists are just two patterns. For a directly installed playwright-core: */node_modules/playwright-core/lib/coreBundle.js. For a nested install via playwright: */node_modules/playwright/node_modules/playwright-core/lib/coreBundle.js. Hitting these two patterns directly with globs down to two levels of repository depth brought the runtime to 0.95 seconds.

Globs are fast because filesystem traversal collapses to OS directory-entry lookups. Where find walks every node, a glob only checks "does an entry matching this pattern exist?" If your search space fits a known structure, glob is clearly superior as a find replacement.

Forgot the Ad-Hoc Signature, Got exit 137

My first implementation had cp -R and the PlistBuddy rewrite, but no codesign. When I tried to launch Chrome Automation.app in a test, the helper processes died instantly with exit 137.

Exit 137 is SIGKILL (128 + 9). The process didn't exit on its own; the OS forcibly killed it. When macOS Library Validation determines that "the binary exists but its signature doesn't match the bundle ID in Info.plist," it SIGKILLs the helper. Google's signature was issued for com.google.Chrome, and the moment you change CFBundleIdentifier in Info.plist to com.google.ChromeAutomation, it's invalid.

Adding codesign --force --deep --sign - and re-running made it launch. --force overwrites the existing signature, --deep re-signs not just the main binary but all frameworks, helpers, and plugins, and - (minus) specifies an ad-hoc signature without an Apple certificate. Gatekeeper doesn't "trust" this signature, but it will launch for local execution. The first time, you get a "developer cannot be verified" confirmation dialog, but after pressing "Open Anyway" once under System Settings → Privacy & Security, it's no longer needed.

Zero Results Still Exited 0 as Success

Early on, I misconfigured SCAN_ROOTS. I wrote ~/Development instead of ~/dev, and since the directory didn't exist, list_core_bundles returned nothing.

The script ran to completion and returned exit 0. patched=0 and already=0, but neither condition tripped, and the log just said playwright patched=0 already=0 rebuilt=0. launchd only looks at the job's exit code, so it kept recording "success" every 6 hours while nothing was being fixed.

This is where the worst case happens: after npm install reverts the path, it can sit unrepaired for up to 6 hours. The code that explicitly fails with exit 1 when patched + already == 0 was born from this failure.

if [ $((patched + already)) -eq 0 ]; then
  log "ERROR: playwright-core が1件も見つからない。SCAN_ROOTS を確認せよ。"
  exit 1
fi

When the exit code is non-zero, launchd records it to StandardErrorPath and re-runs on the next cycle. Make failures loud. That's the correct design.

One Deeply Nested Repo Slipped Through

The first glob pattern only covered one level.

"$root"/*/node_modules/playwright-core/lib/coreBundle.js

This single level doesn't catch ~/dev/social-autolike/node_modules/playwright/node_modules/playwright-core/lib/coreBundle.js. That's the case where playwright is installed containing playwright-core inside it. social-autolike had installed playwright directly, so this nesting occurred.

As a result, while the other 15 locations pointed at Chrome Automation.app, social-autolike alone kept directly hitting Google Chrome.app. When the same problem happened later in dokuji-rosen-sns (described below), the habit of first checking "has a new nesting pattern appeared?" came from this failure.

preflight Was `pkill -9`-ing the Human's Chrome Too

metrics-hub had a script called preflight_chrome.sh whose job was to clean up stale Chrome processes before a job started. It was implemented like this.

pkill -9 "Google Chrome"

pkill does partial matching on the process name. The string Google Chrome appears in the process names of both com.google.Chrome and com.google.ChromeAutomation. Every time an automation job ran preflight, every Chrome window the human had open was SIGKILLed.

Worse, this breaks in a way that's hard to trace back to a cause. Jobs using Chrome Automation.app run normally. The only remaining symptom is that the human's Chrome suddenly crashes. Nothing shows up in the job logs. Only after the vague awareness of "Chrome keeps crashing" accumulates do you realize preflight is the culprit.

The fix is to filter by binary path instead of process name.

pkill -9 -f "Chrome Automation.app/Contents/MacOS"

-f matches against the process's full command line. By targeting only processes whose command line contains the path Chrome Automation.app/Contents/MacOS, the implementation never touches the human's Chrome.

A 4th Repo Fell Into the Same Hole a Month Later

In the September 1 fix, I changed 16 coreBundle.js locations and 10 direct-invocation files, and committed 8 repositories. At that point, I believed "everything is covered."

On September 11, it came out that tools/lib/browser.mjs in dokuji-rosen-sns was still on channel:'chrome'. Even after social-autolike migrated to the bundled chromium on August 12, dokuji-rosen-sns remained on channel:'chrome'. Only between 8:00 and 10:40 in the morning, when it competed with other jobs for Chrome, startup exceeded 180 seconds and timed out — and this continued for 5 days. 39 hangs in 5 days, and the 125 likes in the 8:00 slot vanished every day, yet the alert was skimmed past as a single line: "will auto-recover on the next cycle."

This lesson overlaps with the general form I left in the wiki: "A fix to a shared function means nothing unless it propagates to every call site that uses it." Even if you fix the script, as long as another repo written in the same pattern remains, the fix isn't complete.

Having both ~/dev and ~/content in SCAN_ROOTS of chrome-automation-repair.sh is also to avoid missing a repo that exists in only one of them. Checking "does this repo use channel:'chrome'?" every time a new repo is created is a human task. But if the script runs every 6 hours and patched becomes 1 or more, at least the scenario of "a month passes without noticing a miss" is prevented. For now, as long as the script keeps returning patched=0 already=N, I judge that everything is pointing the right way.

Pitfalls

Here's a complete list of the landmines I actually stepped on. Most of "I set it up but it doesn't work" is one of these.

Forgetting codesign after rewriting Info.plist. The moment you change CFBundleIdentifier to com.google.ChromeAutomation, Google's signature is invalid. Launch without re-signing and Library Validation rejects it; helper processes die instantly with exit 137 (SIGKILL). The --deep in codesign --force --deep --sign - is mandatory. With --force alone, only the main binary is updated, leaving frameworks, helpers, and plugins with the old signature.
Full-walking node_modules with find. find ~/dev ~/content -name "coreBundle.js" took 4 minutes 15 seconds to finish, because there are several hundred node_modules under ~/dev. coreBundle.js only exists in two layouts: */node_modules/playwright-core/lib/ and */node_modules/playwright/node_modules/playwright-core/lib/. Hit those directly with globs and the runtime drops to 0.95 seconds. There's no need to full-walk a known structure.
Writing only one nesting pattern. Projects that install playwright directly have */node_modules/playwright-core/lib/coreBundle.js. Projects where playwright-core comes in via playwright have the nested form */node_modules/playwright/node_modules/playwright-core/lib/coreBundle.js. Write only one in the glob and you miss the other kind of repo every time. I hit this in social-autolike: I thought I'd fixed 16 locations, but one was still pointing at the old path.
Zero results still exits 0 as success. Put a nonexistent path in SCAN_ROOTS and list_core_bundles returns nothing, exiting normally with patched=0 already=0. launchd records exit 0 as "success," and a "do-nothing job" quietly runs every 6 hours. Only by explicitly failing with exit 1 when patched + already == 0 does an ERROR land in the log and the job re-run on the next cycle.
pkill -9 "Google Chrome" kills the human's Chrome too. Partial matching on the process name hits both com.google.Chrome and com.google.ChromeAutomation. preflight_chrome.sh in metrics-hub was exactly this: every time it tried to clean up the automation bundle, it SIGKILLed every tab the human had open. Filter on the full command line with pkill -f "Chrome Automation.app/Contents/MacOS" and the implementation never touches the human's Chrome.
launchd's shell doesn't read ~/.zshrc. The PATH of the shell launchd starts is only /usr/bin:/bin. Neither codesign nor PlistBuddy in /opt/homebrew/bin is visible. Unless you explicitly set PATH, HOME, and LANG in the plist's EnvironmentVariables, the script silently dies with "command not found." If LANG isn't en_US.UTF-8, grep may also mangle filenames containing Japanese. It ends with exit 127 and nothing in the log, so discovery is delayed.
Chrome Automation.app's binary gets read mid-rebuild. cp -R /Applications/Google\ Chrome.app /Applications/Chrome\ Automation.app takes 15–30 seconds in the real environment. If an automation job tries to exec Chrome Automation.app/Contents/MacOS/Google Chrome during that window, it grabs a half-copied binary and crashes. Check for running processes with pgrep -f "Chrome Automation.app/Contents/MacOS", and if any exist, defer to the next 6-hour cycle.
Keychain ACLs are invalidated by Chrome updates. Chrome Automation.app is ad-hoc signed, so its code hash changes with every rebuild. Even if you set "Always Allow" on "Chrome Safe Storage" in macOS Keychain, when Chrome auto-updates and a rebuild runs, the ACL is invalidated and a "Allow password access?" dialog appears on every headless launch. The right answer is to add --use-mock-keychain --password-store=basic to every headless launch. However, persistent profiles like Instagram encrypt session cookies with the Keychain key, so switching to mock wipes the login. That's why keychain_gate() only detects and warns without auto-fixing.
Fixing just one repository means the neighbor hits the same hole a month later. Scripts that directly specify channel:'chrome' are scattered across every repository in ~/dev and ~/content. Fix one, and another repository's browser.mjs keeps pointing at the old path. dokuji-rosen-sns actually hit this: only between 8:00 and 10:40 in the morning, competing with other jobs for Chrome, it hung for over 180 seconds — for 5 days straight. For the record, 39 times in 5 days, with the 125 likes in the 8:00 slot vanishing every day, and the alert was skimmed as a single line: "auto-recovers on next tick." A fix is only one complete unit once you've cross-checked every repository in SCAN_ROOTS.
Unless you look at both StandardOutPath and StandardErrorPath, it looks like "nothing is happening." launchd writes stdout and stderr to separate files. If an error only appears in .err.log and you only check .log, you won't notice. Build the habit from the start of checking both tail -f ~/.claude/logs/chrome-automation-repair.log and tail -f ~/.claude/logs/chrome-automation-repair.err.log, and discovering silent failures shrinks from hours to minutes.
A Gatekeeper dialog appears on Chrome Automation.app's first launch. Since an ad-hoc signature isn't on Apple's trust chain, the first time you get "cannot be opened because the developer cannot be verified." Press "Open Anyway" once under System Settings → Privacy & Security and it won't appear again. Easy to forget right after moving the environment to a new Mac or after a major Chrome update (when the bundle is rebuilt).

Best Practices

Here's the setup above distilled into general principles. Not limited to Chrome bundle isolation — these are design guidelines for any long-running automation on macOS.

1. Separate human resources from unattended-job resources at the level of their "name."
Bundle ID, profile path, port number, PID file. The moment any one of these is shared, you get a structure where "killing one kills the other." Just split the names and the OS isolates them for you. The design cost is one-time; the OS manages it afterward.

2. Make "npm install will always revert it" a design premise.
Don't rely on the fact that a patch was applied. Build "anticipate the revert and re-apply periodically" into the design from the start. A window of up to 6 hours where it reverts to the old path remains, but that's an acceptable known risk. "Fixed" and "stays fixed" are different states.

3. Count patch results and make zero fail-loud.
Keep two variables, patched=N already=M, and exit 1 if patched + already == 0. "Searched but found nothing" is equivalent to "did nothing." Silently exiting 0 means a misconfiguration quietly repeats every 6 hours.

4. Ensure idempotency with version comparison and skip unnecessary rebuilds.
Compare src_ver and dst_ver via /usr/libexec/PlistBuddy -c "Print :CFBundleShortVersionString" and implement "same version, do nothing," and the 6-hour cycle becomes nearly free. A rebuild only runs when Google Chrome auto-updates.

5. Put a pgrep guard before rebuilding.
Check for running processes with pgrep -f "Chrome Automation.app/Contents/MacOS", and if any exist, defer to the next cycle. The problem of a binary being grabbed mid-copy is completely prevented by this guard alone. A mechanism that can decide "don't touch it right now" is what creates long-running stability.

6. Always write PATH, HOME, and LANG in the launchd plist's EnvironmentVariables.
launchd reads none of your interactive shell's configuration. Most cases of "the script works locally but silently dies under launchd" are this. Just writing them in the plist from the start brings this class of trouble to zero.

7. Use glob over find and hit known layouts directly.
If you control the structure of what you're searching, a full walk is always overkill. playwright-core lives in only two patterns. Hitting them directly with globs finishes in 1/270th the time of find.

8. Use codesign with the three-option set --force --deep --sign -.
--force overwrites the existing signature, --deep applies to all frameworks, helpers, and plugins, - is an ad-hoc signature. Miss even one and Library Validation SIGKILLs. Remember them as a set of three and you won't get stuck.

9. Add --use-mock-keychain --password-store=basic to headless Chrome.
Only jobs with persistent profiles (Instagram etc.) are the exception: using mock wipes the session, so keep the real Keychain for them. Know which category applies, and for the majority of headless launches, adding mock is the correct setting.

10. Use -f with pkill for full-path matching. Don't use partial process-name matching.
pkill "Google Chrome" kills both even though the bundle IDs differ. Filter on the full command line with -f "Chrome Automation.app/Contents/MacOS" and the implementation never touches the human's Chrome.

11. After a fix, cross-check every repository in SCAN_ROOTS.
Fixing one repository isn't complete while another repository written in the same pattern remains. Surface the misses with grep -r "channel:'chrome'" ~/dev ~/content --include="*.mjs" --include="*.js" 2>/dev/null, and only after handling all of them is it done.

12. Keep the final verification commands together with the script.
Decide up front what you'll verify with after setup. For this setup, it's these four commands.

# 1. launchd への登録確認
launchctl list | grep chrome-automation-repair

# 2. バンドルが起動できる状態か
open -a "Chrome Automation" --args --version

# 3. playwright が正しいパスを見ているか
grep -r "Chrome Automation" ~/dev/*/node_modules/playwright-core/lib/coreBundle.js 2>/dev/null | head -5

# 4. 人間の Chrome は独立して起動できるか
open -a "Google Chrome"

If number 2's --args --version returns Chrome's version string (e.g. Google Chrome 127.0.6533.120) and number 4 opens without -600, the isolation is working correctly. Thinking of "the setup plus its verification commands as one unit" lowers the cost of reproducing the same environment six months later.

13. Collect logs in ~/.claude/logs/ and check both stdout and stderr.
Look at only one and you'll keep not noticing that an error is in .err.log. Set things up so tail -f ~/.claude/logs/*.log ~/.claude/logs/*.err.log gives you every job's status at a glance, and the cost of discovering silent failures drops.

14. Choose the repair window (SLA) deliberately and record the rationale.
StartInterval: 21600 (6 hours) is an SLA: "even if npm install runs, it's repaired within at most 6 hours." Shorter raises certainty but increases CPU and disk impact. It's important to deliberately choose the maximum your environment can tolerate and leave the rationale in a comment in the plist.

Summary

If I had to state in one line what remained as a design from that September 1 afternoon when open -a "Google Chrome" returned -600: "Contention only happens while resources are shared. Split the name, and the OS isolates them for you."

With one automation, the problem is invisible. As you stack up two, five, ten, human resources and automation resources occupy the same namespace, and a structure where killing one breaks the other quietly forms. The more automation you add to grow revenue, the more your own working environment breaks — noticing that paradox was the starting point of this design.

The mechanism stabilized the moment I stopped viewing "npm install reverts it" as a "problem that must be fixed" and accepted it as "periodic repair on the premise that it reverts." Even if the patch reverts, it's fixed within 6 hours. Building that repair window into the design as a known risk is the key to long-running operation.

Now that chrome-automation-repair.sh runs every 6 hours and keeps returning patched=0 already=16 rebuilt=0, the automation jobs use Chrome Automation.app and the human's Chrome runs independently. Mornings like September 1 don't come anymore.

What's the one shared resource in your automation setup that you and your jobs are still fighting over — and how long has it been silently losing?

The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day procedure are compiled in a paid note (Japanese).

📕 How to actually earn with a Claude Code autonomous environment — mechanism, real examples, getting started, and support

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

Wi-Fi Was Dead for 5 Hours While Chrome Kept Working: 4,960 DNS Failures and the 5-Minute Probe That Catches Them

Lily — Fri, 18 Sep 2026 05:00:06 +0000

Last time I wrote about how many jobs you should re-run the moment your quota comes back. This post goes back a few days earlier, to a quieter and far more annoying problem: the morning my home Wi-Fi was effectively dead for five hours — while the browser kept loading pages as if nothing were wrong.

The problem: the browser works, but only the jobs die

On the morning of 2026-09-12, four unattended jobs failed one after another.

Job	Time
`com.lily.es-daily-rows`	05:01
`com.lily.paid-note-pin-guard`	07:20
`com.lily.line-column-gen`	08:40
`com.lily.line-pdca`	08:41

The errors were Node fetch failed and python getaddrinfo failures. Yet during that same window, browsing in Chrome worked perfectly. This is the worst kind of failure: "the network is up, but some processes die anyway."

Digging into the unified log, airportd had been emitting SlowWiFiDnsFailure at an abnormal rate. In the last 24 hours there were 4,960 of them. They ramped up around 05:43, ran at 1,000–1,700+ per hour through the 06:00–11:00 window, peaked at 1,720 in the 10 o'clock hour, and stopped cold after 12:38. The human action that ended it came at 13:09: I manually switched from home Wi-Fi to my phone's tethering. After the switch, 37 probe runs showed failed: 0, with lookups under 60ms and connects under 100ms on both IPv4 and IPv6.

The culprit was that only the DNS path was dead. Chrome has its own DNS resolution (DoH plus a cache), so it sails on even when the home router's DNS is rotten. Node and Python go straight to the system resolver — the home router's DNS — and get stuck. The intuition that "the browser works, so the network is fine" was itself the diagnostic trap.

Pitfalls I hit (diagnosis edition)

I almost misread a symlink's mtime as "we've been on tethering for three weeks" → /etc/resolv.conf is a symlink, and its mtime still said Aug 15. Checking the real file, /var/run/resolv.conf, its mtime matched the switchover time (13:09:39) exactly, so the first reading was wrong. You always have to know whether you're looking at the symlink's mtime or the target's.
Unified log retention windows differ per facility → configd only had entries from 05:54 onward, while airportd went back to 19:00 the previous day. I thought I was comparing two facilities on the same timeline, but one of them had already dropped data.
The name SlowWiFiDnsFailure is misleading → It's logged as "slow," not "dead," so even grepping for anomalies requires a threshold decision.
Node fetch failed appeared 343 times in that same 24 hours. The apparent gap between the job failure times (05:01–08:41) and the start of the airportd burst (05:43 onward) is also explained by the difference in facility retention.

Note
On that day I was able to dig through the unified log afterward and pin down "roughly 5,000 events between 06:00 and 12:38." But that only worked because the incident happened to be large enough to stand out. If the same thing happens next time for a shorter period or at lower frequency, it may be past the unified log's retention window and leave nothing behind. What I needed was a way to trace root causes that doesn't depend on "digging through the log after the fact" — and that's the real subject of this post.

Never chasing it after the fact again: net-probe.sh

What I built is a script that does one thing: every 5 minutes it appends a one-line JSONL "snapshot of the network state." It uses neither Chrome nor claude, and the target is to finish in under 20 seconds.

#!/bin/bash
# 5分毎の軽量ネットワークプローブ（Chrome/claude 不使用・目標20秒以内)。
# gateway / SSID / nameserver / dig 3ホスト / curl -4,-6 2ホスト / python getaddrinfo を
# 1行JSONで ~/.claude/logs/net-probe.jsonl へ追記する。
# 目的: Node fetch failed / python getaddrinfo 失敗が「どの回線・どのDNSで」起きたかを後から突合する。

The key is to fire everything in parallel. Fetching the SSID via system_profiler alone takes 5 seconds, so it gets pushed into the background alongside the other collection.

# --- SSID は system_profiler が5秒かかるので並列 ---
( system_profiler SPAirPortDataType 2>/dev/null | awk '/Current Network Information:/{getline; gsub(/^ +| *:$/,""); print; exit}' > "$TMP/ssid" ) &

# --- DNS: システム順序で3ホスト + 各NSで oauth2 (すべて並列, 2秒×1回) ---
for h in oauth2.googleapis.com discord.com note.com; do
  ( dig +time=2 +tries=1 "$h" A > "$TMP/dig.$h" 2>&1 ) &
done
i=0
for ns in ${NS//,/ }; do
  ( dig +time=2 +tries=1 @"$ns" oauth2.googleapis.com A > "$TMP/digns.$i" 2>&1; echo "$ns" > "$TMP/digns.$i.ns" ) &
  i=$((i+1))
done

dig uses +time=2 +tries=1 so it always gives up at 2 seconds, curl uses --max-time 6, and Python's getaddrinfo runs in a thread with join(5) to force a 5-second cutoff. Every timeout has to be explicit, because when DNS is truly dead, the probe itself would hang and miss the next 5-minute slot.

def f():
    t = time.time()
    try:
        r = socket.getaddrinfo('oauth2.googleapis.com', 443)
        res.update(ok=True, n=len(r), ms=round((time.time()-t)*1000))
    except Exception as e:
        res.update(ok=False, err=type(e).__name__+': '+str(e), ms=round((time.time()-t)*1000))
th = threading.Thread(target=f, daemon=True); th.start(); th.join(5)
if not res: res.update(ok=False, err='timeout>5s', ms=5000)

Once wait confirms all the fragments are in, the one-line JSON is assembled in Python rather than bash. Hand-building JSON via string concatenation breaks the moment an SSID contains a space or a special character, so this one part is left to json.dumps.

gw = os.environ['GW']
fail = sum(1 for d in dns.values() if d['status'] != 'NOERROR') + sum(1 for h in http.values() if h['http'] == 0) + (0 if py.get('ok') else 1)
rec = {
  'ts': os.environ['TS'], 'gateway': gw, 'gateway6': os.environ['GW6'], 'iface': os.environ['IFACE'],
  'ip4': os.environ['IP4'], 'ssid': rd(f'{T}/ssid').strip(), 'security': os.environ['SEC'],
  'isIphoneHotspot': gw == '172.20.10.1', 'resolvers': [x for x in os.environ['NS'].split(',') if x],
  'utun': int(os.environ['UTUN'] or 0), 'dns': dns, 'dnsPerResolver': per_ns, 'http': http,
  'pyGetaddrinfo': py, 'failures': fail,
}

isIphoneHotspot is determined by whether the gateway IP is 172.20.10.1 (the default address for iPhone tethering). Instead of eyeballing the gateway IP every time, this single field tells you at a glance which connection you're on. failures is the sum of dns / http / pyGetaddrinfo failures, so pulling anomalous lines out of the JSONL is just grep '"failures":[1-9]'.

launchd: run it quietly every 5 minutes

StartInterval gives a fixed-interval launch, and the priority is lowered so it stays strictly in the background.

<key>StartInterval</key>
<integer>300</integer>
<key>LowPriorityIO</key>
<true/>
<key>Nice</key>
<integer>10</integer>
<key>ProcessType</key>
<string>Background</string>

Nice 10 plus LowPriorityIO ensures it never steals CPU or IO from the other automation jobs. A job whose purpose is to watch whether the network is alive should not itself become a source of resource contention.

Four days of real measurements

From deployment until today, it has accumulated 662 lines.

$ wc -l ~/.claude/logs/net-probe.jsonl
662

Broken down by gateway: home Wi-Fi (192.168.3.1) accounts for 594 entries, iPhone tethering (172.20.10.1) for 68. Lines matching "failures":[1-9] number 50 — about 7.6% of 662. Some days had zero, while the hour 2026-09-13T09 alone had 5 failures clustered together — unevenness that would have been flattened out without 5-minute granularity.

A single line looks like this (excerpt from the actual log).

{"ts":"2026-09-16T20:28:45+0900","gateway":"192.168.3.1","iface":"en0",
 "security":"WPA2_PSK","isIphoneHotspot":false,
 "dns":{"oauth2.googleapis.com":{"status":"NOERROR","ms":17,"answers":5,"err":null},
        "discord.com":{"status":"NOERROR","ms":17,"answers":11,"err":null},
        "note.com":{"status":"NOERROR","ms":15,"answers":8,"err":null}},
 "http":{"v4:discord.com":{"http":200,"lookup":0.004285,"connect":0.013422,"total":0.115357}, ...},
 "pyGetaddrinfo":{"ok":true,"n":4,"ms":107},"failures":0}

Cross-referencing: reconstructing when, which connection, and which DNS after the fact

The network-flap-probe-2026-09-12.json I put together right after the incident still holds verdict: "dns-resolver-flap" along with the explanation of the cause, exactly as written that day (in Japanese).

"verdictDetail": "症状の発生源は自宅Wi-Fiの DNS 経路。airportd が 06:00-12:38 に
SlowWiFiDnsFailure を約5,000件（毎時1,000-1,700件）記録しており、failing 4 ジョブと
fetch failed 343件はこの回線上で発生。13:09に人間が iPhone テザリングへ手動切替した後は
fault 0・全プローブ成功。Chrome 経由ジョブが通っていたのは Chrome が独自 DNS
(DoH/非同期リゾルバ+キャッシュ)を使い、Node/pythonはシステムリゾルバ
（自宅ルータ DNS）を直撃するため"

In short, it says: the source of the symptoms was the home Wi-Fi's DNS path; airportd recorded roughly 5,000 SlowWiFiDnsFailure events (1,000–1,700 per hour) between 06:00 and 12:38; the 4 failing jobs and the 343 fetch failures all occurred on that connection; after the manual switch to iPhone tethering at 13:09, faults dropped to 0 and every probe succeeded; and the Chrome-based jobs kept working because Chrome uses its own DNS (DoH / async resolver + cache) while Node and Python hit the system resolver (the home router's DNS) directly.

That day I dug all the way there by hand. With net-probe.jsonl, the same conclusion can be reached mechanically, at 5-minute granularity. The gateway field records the connection, and the dnsPerResolver field records how each individual DNS server responded to its query, so "when, on which connection, against which DNS server" can be reconstructed just by filtering fields. Whether a manual reproduction step (rebooting the router, pinning DNS to 1.1.1.1/8.8.8.8) is even needed can be decided case by case from this record.

Summary

Wi-Fi can be "connected" while only the DNS path is dead. Chrome carrying on as normal is because it has its own DNS cache — it is not evidence of health.
If root-cause tracing depends on "digging through the unified log after the incident," you run into traps like per-facility retention differences and misreading a symlink's mtime.
net-probe.sh is a thin mechanism that fires gateway / SSID / DNS / HTTP (v4, v6) / getaddrinfo all in parallel with timeouts and appends one JSONL line in under 20 seconds.
With launchd's StartInterval=300 + Nice 10 + LowPriorityIO, it runs quietly every 5 minutes without competing with other jobs.
662 lines in 4 days, 50 of them with failures>0 (7.6%). With this accumulated, the next time the same thing happens, reaching the cause means "filter the fields," not "dig after the fact."

Using this JSONL, I'd like to grow it to the point where a detected anomaly automatically notifies Discord and, if needed, switches connections automatically. That's for another post.

Have you ever had a job fail while the browser insisted everything was fine — and how long did it take you to find out DNS was the reason?

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

15 launchd Jobs and One Quota Circuit Breaker: Deciding What to Re-run Once the Circuit Closes

Lily — Fri, 18 Sep 2026 00:00:03 +0000

A circuit breaker that halts every job the moment your quota runs dry is only half the story. The harder question shows up afterward: once the circuit closes again, what do you do with everything it skipped? In my previous post I wrote about fixing the Wiki secret-scan sync. This time I'm switching gears and following up on claude-quota-guard.py, the quota circuit breaker I built. That script's story ended at "detect quota exhaustion, stop all jobs." In real operation, there's a whole second problem waiting past that point—and along the way it involved a 1200-second timeout that turned out to need 2700, and a load average that climbed past 40 when I got it wrong.

The problem: the circuit closes, but skipped jobs don't come back

run_job in claude-quota-guard.py returns exit 0 immediately when a job starts while the circuit is open, leaving nothing but a marker in the log.

print(
    "CLAUDE_QUOTA_JOB_SKIPPED "
    f"job={label} reason={status['reason']} remaining={status['remaining_seconds']}s ts={now()}",
    file=sys.stderr,
)
return 0

This guard sits in front of 15 of the plists under ~/Library/LaunchAgents/*.plist. As the comment in the code puts it:

# 🔴 2026-08-21: circuit が開くと全ジョブが一律で止まるため、消費の大半を占める
# 返信/エンゲージ系がクォータを使い切った巻き添えで「投稿」まで停止していた。

The problem is that after the circuit closes and open_until has passed, launchd does nothing until that job's next StartCalendarInterval slot comes around. If the 9:00 AM job was skipped for quota reasons and the quota recovers at 10:00, but the next slot is 9:00 AM tomorrow, you've lost an entire day's execution. Deciding "what to re-run, and how much, after recovery" is the job of quota-catchup.py.

Design: three filters decide the candidates

quota-catchup.py walks every plist in find_candidates and only treats a job as a re-run candidate if it passes three conditions ANDed together.

Condition	Function	Purpose
Is this plist guarded?	`is_guarded`	Don't mix in unrelated jobs
Is today's latest marker SKIPPED?	`latest_marker_is_today_skip`	Exclude jobs that already ran, and stale skips
Has a slot already passed?	`calendar_slot_passed`	Exclude `StartInterval` jobs and jobs with only future slots

def find_candidates(...) -> list[Candidate]:
    candidates: list[Candidate] = []
    for plist_path in sorted(launch_agents.glob("*.plist")):
        plist = load_plist(plist_path)
        if not plist or not is_guarded(plist):
            continue
        ...
        if label in already_kicked or not latest_marker_is_today_skip(output_paths, label, today):
            continue
        if calendar_slot_passed(plist, now):
            candidates.append(Candidate(label, plist_path))
    return candidates

Preventing double launches: is_guarded

Re-run targets are limited to jobs that are "launched via claude-quota-guard.py." The check is nothing more than a string match on ProgramArguments.

def is_guarded(plist: dict) -> bool:
    arguments = plist.get("ProgramArguments")
    return isinstance(arguments, list) and any("claude-quota-guard" in str(value) for value in arguments)

If you picked up plists that don't go through the guard, you'd end up kicking ordinary cron-style jobs that have nothing to do with the quota.

Deciding "has a slot already passed?": calendar_slot_passed

This is the star of the show. A single job can have multiple StartCalendarInterval slots. A real example is com.shun.daily-brief.plist.

<key>StartCalendarInterval</key>
<array>
    <dict><key>Hour</key><integer>8</integer><key>Minute</key><integer>0</integer></dict>
    <dict><key>Hour</key><integer>10</integer><key>Minute</key><integer>30</integer></dict>
</array>

Even if the 8:00 slot gets skipped for quota, the same job naturally runs again at 10:30. So the rule is "if even one slot has passed, it's a re-run candidate," not "wait, because there's still a future slot." Conversely, StartInterval jobs (e.g., every 30 minutes) will naturally re-run on the next interval if you just leave them alone, so there's no need to make them catch-up targets.

def calendar_slot_passed(plist: dict, now: datetime) -> bool:
    """True only for calendar-only jobs with at least one past slot today."""
    if "StartInterval" in plist:
        return False
    raw_entries = plist.get("StartCalendarInterval")
    if isinstance(raw_entries, dict):
        entries = [raw_entries]
    elif isinstance(raw_entries, list):
        entries = raw_entries
    else:
        return False

    saw_today_slot = False
    for entry in entries:
        if not isinstance(entry, dict) or "Hour" not in entry:
            return False
        if not runs_today(entry, now):
            continue
        try:
            scheduled = now.replace(
                hour=int(entry["Hour"]),
                minute=int(entry.get("Minute", 0)),
                second=0, microsecond=0,
            )
        except (TypeError, ValueError):
            return False
        if scheduled > now:
            continue
        saw_today_slot = True
    return saw_today_slot

The key is walking every slot to the end and accumulating saw_today_slot with an OR. If you'd written it as "return the verdict from the first slot you find," you'd hit a bug where the result depends on the order of the slots. For example, with the ordering [{9:00}, {18:00}] evaluated at 13:00, the correct answer is "candidate" because 9:00 is in the past—but if the loop only judged by the last entry, it would misclassify the job as not a candidate on the grounds that 18:00 is in the future.

A timeout tuned from real measurements: JOB_TIMEOUT_SECONDS

A re-run goes through kick_and_wait, which runs launchctl kickstart and then polls until the job leaves launchd's management (i.e., exits). This timeout started at 1200 seconds, which turned out to be insufficient in practice.

# 実測(2026-08-21): affameba-gen 等の claude 生成レーンは 20 分を超える。
# 1200s だと「待つのをやめて次を kick」するだけで前のジョブは生き続け、
# runbook が要求する直列 kick が崩れて重い生成が重なる（load 40 超の二次被害）。
JOB_TIMEOUT_SECONDS = 2700
# timeout 時は待つのをやめるだけでなく実際に止める。ここを殺さないと直列性が保てない。
JOB_KILL_GRACE_SECONDS = 30

The lesson here: "stop waiting" and "stop the job" are two different things. With only the former, the timed-out old job kept running in the background while the next candidate got kicked, the generation workloads piled up, and the load average went past 40 as collateral damage. So terminate_job actually kills the job—SIGTERM, then SIGKILL—to guarantee serial execution.

def terminate_job(domain_label: str, label: str) -> None:
    """timeout したジョブを実際に止める。次の kick と重ならせないための直列性の担保。"""
    for signal_name in ("SIGTERM", "SIGKILL"):
        subprocess.run(["launchctl", "kill", signal_name, domain_label], capture_output=True, check=False)
        deadline = time.monotonic() + JOB_KILL_GRACE_SECONDS
        while time.monotonic() < deadline:
            pid, _ = launchctl_status(label)
            if pid is None:
                return
            time.sleep(2)

Pinning down the edge cases with tests: test_quota_catchup.py

This script has 18 test cases (17 in unittest, plus 1 pytest-style function). Writing this many tests for a personal automation script might look like overkill, so here are the cases where they actually earned their keep.

Don't call claude on days with zero candidates

This is the one with the biggest real-world cost. The comment in the run function explains why.

# 拾うものが無い日に probe を撃つと、30分おきに claude -p を1日48回空撃ちして
# クォータを削る（このジョブが防ごうとしている事故そのものを起こす）。
# 候補が出た時だけ回復を確認する。
if not candidates:
    return [], 0, 0

The test that protects this is test_no_candidates_skips_probe.

def test_no_candidates_skips_probe(self):
    self.add_job(
        error_text=f"CLAUDE_QUOTA_JOB_RAN job=com.lily.test exit=0 ts={int(self.now.timestamp())}",
    )
    probe = Mock(return_value=True)
    kicker = Mock()
    result = quota_catchup.run(
        dry_run=False, now=self.now, state_path=self.state, catchup_path=self.catchup,
        launch_agents=self.agents, log_path=self.root / "result.log", probe=probe, kicker=kicker,
    )
    self.assertEqual(result, ([], 0, 0))
    probe.assert_not_called()
    kicker.assert_not_called()

"The batch that checks whether the quota has recovered burns quota just by checking" is a self-contradiction that anyone who built a circuit breaker absolutely does not want to step into. That single line, probe.assert_not_called(), guarantees it mechanically.

Today's skip is a candidate; a skip from three days ago is not

def test_today_skip_is_candidate(self):
    label = self.add_job()
    self.assertEqual([item.label for item in self.candidates()], [label])

def test_old_skip_is_not_candidate(self):
    self.add_job(mtime=self.now - timedelta(days=3))
    self.assertEqual(self.candidates(), [])

latest_marker_is_today_skip uses a regex to pull ts= out of the log and checks whether the date is today. If a SKIPPED log from three days ago got picked up again today, you'd have a zombie state where past failures get re-run every single day.

A past slot makes it a candidate even if a future slot remains

def test_past_slot_makes_candidate_even_if_later_slot_is_future(self):
    self.add_job(schedule=[{"Hour": 9, "Minute": 0}, {"Hour": 18, "Minute": 0}])
    self.assertEqual([item.label for item in self.candidates()], ["com.lily.test"])

This test pins the OR logic in calendar_slot_passed described above, using a real daily-brief-style schedule (multiple slots).

Don't get the marker order wrong

The log contains both SKIPPED and RAN. If you misjudge which one is the latest marker, you'll either double-kick a job that actually succeeded, or miss a separate skip that happened after a success.

def test_ran_marker_after_skip_in_same_log_is_not_candidate(self):
    label = "com.lily.mixed"
    today = int(self.now.timestamp())
    self.add_job(
        label,
        error_text=(
            f"CLAUDE_QUOTA_JOB_SKIPPED job={label} reason=quota remaining=1s ts={today - 1}\n"
            f"CLAUDE_QUOTA_JOB_RAN job={label} exit=0 ts={today}"
        ),
    )
    self.assertEqual(self.candidates(), [])

If RAN comes after SKIPPED, the job "succeeded later after all" and is not a candidate. latest_job_marker guarantees this chronological judgment by scanning reversed(lines).

Note
The common goal of this script's unit tests is not "prove the clever logic is correct" but "pin down, ahead of time, the boundaries a naive implementation gets wrong." The OR logic in calendar_slot_passed, the old-vs-new marker judgment, suppressing the probe when there are zero candidates—each of these flips its result depending on how a single line is written, and none of them are easy to notice until you actually run it. The value of writing pytest for a personal automation script isn't to convince a reviewer; it's so that six months from now, when you change the spec, you don't step on the same mistake again.

Pitfalls I hit

A 1200-second timeout cut off jobs that run over 20 minutes → Extended to 2700 seconds based on measurements, with JOB_KILL_GRACE_SECONDS=30 providing the SIGTERM→SIGKILL grace period
Merely "stop waiting" on timeout breaks serial execution → Without actually killing via terminate_job, the previous job stays alive while the next one gets kicked, causing the load-over-40 collateral damage
Firing the probe on a day with zero candidates causes the very accident you're trying to prevent → Only call probe_claude() when there are candidates
Picking up StartInterval jobs causes unnecessary re-runs → If StartInterval is present, calendar_slot_passed is unconditionally False
Judging SKIPPED/RAN markers by "does it exist" alone leads to misclassification → Look only at the latest marker via reversed(lines)
Hitting claude on every --dry-run turns verification cost into execution cost → dry-run doesn't call the probe; it just returns the candidate list

Summary

A quota circuit breaker doesn't end at "stop"—unless you also design what to re-run after it closes, recovery waits until the next day
Re-run candidates are the three-stage AND of is_guarded / latest_marker_is_today_skip / calendar_slot_passed. If even one slot has passed, it's a candidate; whether future slots exist is irrelevant
Set the timeout from real measurements (jobs over 20 minutes actually exist), and when you cut a job off, actually kill it to preserve serial execution
With zero candidates, don't even call the probe. Avoid the self-contradiction of a recovery-check batch eating its own quota
Unit tests for personal scripts exist to pin down in advance the boundaries that naive implementations tend to flip (slot order, marker recency, side effects on zero-count days)

Next time I plan to write about the circuit breaker itself—how claude-quota-guard.py distinguishes a genuine "limit reached" message from a successful run whose article body just happens to contain the same words.

How do you handle catch-up for scheduled jobs that got skipped in your own setup—do you re-run them, or just wait for the next slot?

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

4 Ways a 'Retired' launchd Job Kept Running on a 160+ Job Fleet — and the 3 Commands That Actually Stop It

Lily — Thu, 17 Sep 2026 11:00:06 +0000

I run more than 160 launchd jobs on a Mac that almost never reboots. That fleet is what carried me from a ¥100k/month student side gig to ¥600k juggling multiple jobs, back down to zero after a layoff, and then — six months of rebuilding a Claude Code autonomous setup later — past ¥1.2M/month in revenue. Keeping that automation alive has taught me something I didn't expect: cleanly stopping a job is far harder than starting one.

Renaming a file to .plist.retired does not stop the job. launchd keeps running it.

Why this happens

launchd doesn't identify jobs by file name. It identifies them by the value of the Label key inside the plist.

Take a file that actually sits in my environment in the .retired state: ~/Library/LaunchAgents/com.shun.zenn-daily.plist.retired.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.shun.zenn-daily</string>
  <key>ProgramArguments</key>
  <array>
    <string>/bin/bash</string>
    <string>~/.discord/run-and-notify.sh</string>
    <string>zenn</string>
    <string>Zenn日次公開</string>
    <string>/bin/bash</string>
    <string>~/.claude/scripts/zenn-daily-publish.sh</string>
    <string>apply</string>
  </array>
  <key>StartCalendarInterval</key>
  <array>
    <dict><key>Hour</key><integer>7</integer><key>Minute</key><integer>30</integer></dict>
    <dict><key>Hour</key><integer>10</integer><key>Minute</key><integer>0</integer></dict>
    <dict><key>Hour</key><integer>14</integer><key>Minute</key><integer>0</integer></dict>
    <dict><key>Hour</key><integer>19</integer><key>Minute</key><integer>0</integer></dict>
  </array>
  <key>RunAtLoad</key>
  <false/>
  <key>ProcessType</key>
  <string>Background</string>
</dict>
</plist>

The file name is com.shun.zenn-daily.plist.retired, but the Label inside is still com.shun.zenn-daily. launchd already read this file at login and registered it in its internal table under the label com.shun.zenn-daily. Renaming the file afterwards does nothing to that table entry.

This job has four times in StartCalendarInterval: 7:30, 10:00, 14:00, and 19:00. After the rename, launchd keeps calling ~/.claude/scripts/zenn-daily-publish.sh apply at every one of those times. Check the logs and you'll find execution traces piling up in ~/.claude/logs/zenn-daily.out.log and ~/.claude/logs/zenn-daily.err.log.

launchd's ID is the Label, not the path

macOS launchd is designed differently from Linux systemd. systemd ties the unit file path tightly to the service name, whereas launchd treats the plist path as nothing more than "where to load from the first time." Everything after that is managed entirely by the Label value.

There's a rational design reason for this. launchd sits close to the core of macOS, and it needs to detect conflicts when jobs with the same Label are scattered across multiple locations (~/Library/LaunchAgents/, /Library/LaunchAgents/, /Library/LaunchDaemons/). The flip side: when a file disappears from LaunchAgents/, there is no mechanism that automatically removes the already-loaded entry.

Why it keeps running after the rename

macOS watches ~/Library/LaunchAgents/ as a directory. When a file is added, launchd reads and loads it; when one is removed, that triggers an unload. But a rename is reported as "deleted under the old name, added under the new name."

Here's the catch. Even if com.shun.zenn-daily.plist is deleted, what launchd would unload is "the job whose Label was written in that file." On macOS 13 and later, when deciding whether com.shun.zenn-daily.plist.retired should be treated as a plist, files whose extension isn't .plist are excluded from watching and ignored. In other words:

com.shun.zenn-daily.plist → delete event → should be unloaded in principle
But within an already-loaded session, there are cases where auto-unload via filesystem watching doesn't take effect

You may get lucky and have it unloaded at the moment of the rename, but across a login session boundary it will definitely still be there. If you leave the file in the .retired state and reboot, it never gets loaded in the first place, so no problem appears. But on a Mac that has been running for a long time without a reboot, a rename alone means the job absolutely keeps running. As your automation grows, that Mac reboots less and less often. That's why "it's supposed to be retired but it's running" is such an easy trap to fall into.

What the damage looks like

What actually happened in my environment: "a lane that was supposed to be retired kept running on a different scheduler, and the day after I moved its working folder, it started firing blanks." After the folder move, execution simply fails, so exit 1 piles up in the logs — but from launchd's point of view it's just "the job ran (and failed)," so nothing gets unloaded. A pattern of quiet failures that keep going.

When you operate a fleet of 160+ jobs, several of these "ghost jobs" can be mixed in at once. Unless you periodically take inventory with launchctl list | grep com.lily, unintentionally leftover jobs keep consuming resources that other jobs need (AVD locks, browser slots, quota).

The overall flow

Retiring a launchd job correctly takes three steps. File operations come afterwards — getting bootout through comes first.

┌─────────────────────────────────────────────────┐
│  plist.retired にリネームした状態（NG）            │
│                                                 │
│  ~/Library/LaunchAgents/                        │
│    com.shun.zenn-daily.plist.retired  ← ファイル名変更済み │
│                                                 │
│  launchd 内部テーブル                             │
│    Label: com.shun.zenn-daily  ← まだ生きている   │
│      → 7:30 / 10:00 / 14:00 / 19:00 に発火      │
└─────────────────────────────────────────────────┘

            ↓ bootout を実行する

┌─────────────────────────────────────────────────┐
│  正しい退役後の状態（OK）                           │
│                                                 │
│  ~/Library/LaunchAgents/                        │
│    com.shun.zenn-daily.plist.retired            │
│                                                 │
│  launchd 内部テーブル                             │
│    Label: com.shun.zenn-daily  → エントリなし     │
│      → 発火しない                                 │
└─────────────────────────────────────────────────┘

Step 1: Unload with bootout

launchctl bootout gui/$UID/com.shun.zenn-daily

gui/$UID is the login session domain. $UID expands to the current user's UID as-is. If you get a No such process error, ignore it and continue. That just means you called bootout on a job that was already unloaded — not an anomaly.

There's an important pitfall. If you bootout a running job and immediately try to bootstrap (reload) it, it can fail with 5: Input/output error. This is a race condition where the next command runs before the kernel has finished tearing down the service. In my environment, com.lily.threadspilot.engage vanished once because of this. When you do need to reload, insert a script that retries up to 15 times at 1-second intervals, or sleep 2 before bootstrap.

The goal here is only to "stop," so no bootstrap is needed.

Step 2: Confirm it's gone with launchctl list

launchctl list | grep zenn-daily

If the output is empty, the entry is gone from launchd's table. The job will not fire.

However, not appearing in launchctl list is not a sufficient condition for "retirement complete." The reverse problem matters more: even if a job appears in launchctl list, that only means it's "registered" — not that it's "loaded with the latest plist definition."

I fell into this trap when I bulk-injected a quota-guard wrapper into 44 jobs. In launchctl list, every job looked registered, but when I actually checked the contents with launchctl print, several jobs still had the old ProgramArguments. Only the live definition returned by launchctl print is trustworthy evidence of state.

For retirement checks, disappearing from list is enough; but for post-reload checks, make it a habit to always use print.

# 退役確認用（消えていればOK）
launchctl list | grep com.shun.zenn-daily

# 再ロード後の定義確認用（新しいProgramArgumentsが反映されているか）
launchctl print gui/$UID/com.shun.zenn-daily

Step 3: Tidy up the file (optional)

Once bootout goes through, the job won't fire even if the file remains in LaunchAgents/. And since .plist.retired files are outside launchd's watch scope, they won't be auto-loaded at the next login or reboot either.

Leaving it like this causes no operational problem, but it makes the fleet harder to read. My policy is to move retired plists to a backup directory.

mkdir -p ~/content/launchagents-backup-$(date +%Y%m%d)
mv ~/Library/LaunchAgents/com.shun.zenn-daily.plist.retired \
   ~/content/launchagents-backup-$(date +%Y%m%d)/

After moving, confirm there are no leftovers with ls ~/Library/LaunchAgents/ | grep zenn.

Putting it together: the complete retirement command sequence

# 1. アンロード（No such process は無視してよい）
launchctl bootout gui/$UID/com.shun.zenn-daily

# 2. テーブルから消えたか確認
launchctl list | grep zenn-daily
# 出力が空であればOK

# 3. plistをバックアップへ移動
BACKUP=~/content/launchagents-backup-$(date +%Y%m%d)
mkdir -p "$BACKUP"
mv ~/Library/LaunchAgents/com.shun.zenn-daily.plist.retired "$BACKUP/"

# 4. 移動後の残骸確認
ls ~/Library/LaunchAgents/ | grep zenn

These four steps are the answer to "what should I actually have done when I renamed it to .plist.retired?"

Why the `.retired` rename became a habit

The macOS convention of renaming to .plist.disabled or .plist.retired exists because it's "easy to bring back later." Deleting outright makes it expensive to reconstruct the original plist. Keeping it in the same place under a .retired name as a backup is a reasonable idea.

The problem is that, unless you know how launchd behaves, it's hard to see that a rename is a "declaration of intent to retire," not the "execution of retirement." If you managed plists under Git, git log would give you the change history and the backup rename would be unnecessary — but almost nobody puts ~/Library/LaunchAgents/ under Git. So the rename technique lives on.

To state the procedure precisely: the rename is the backup. bootout is the stop. They are two separate operations.

Implementation details

Adding just 4 elements to `ProgramArguments`

The bootout → bootstrap cycle is the same whether you're "stopping" a job or "switching it to run through a wrapper." In August 2026, when I bulk-inserted a quota guard into a fleet of 160+ jobs, this was the entire change.

Before (the actual structure of com.shun.zenn-daily):

<key>ProgramArguments</key>
<array>
  <string>/bin/bash</string>
  <string>~/.discord/run-and-notify.sh</string>
  <string>zenn</string>
  <string>Zenn日次公開</string>
  <string>/bin/bash</string>
  <string>~/.claude/scripts/zenn-daily-publish.sh</string>
  <string>apply</string>
</array>

After (only 4 elements added at the front):

<key>ProgramArguments</key>
<array>
  <string>~/.claude/scripts/quota-guard.sh</string>
  <string>--job</string>
  <string>com.shun.zenn-daily</string>
  <string>--</string>
  <string>/bin/bash</string>
  <string>~/.discord/run-and-notify.sh</string>
  <string>zenn</string>
  <string>Zenn日次公開</string>
  <string>/bin/bash</string>
  <string>~/.claude/scripts/zenn-daily-publish.sh</string>
  <string>apply</string>
</array>

Touch only ProgramArguments. Not a single key among StartCalendarInterval (the 4 slots at 7:30, 10:00, 14:00, 19:00), ProcessType (Background), RunAtLoad (false), StandardOutPath, or StandardErrorPath changes. If you change the schedule and the output destinations at the same time, you can no longer tell "is this a wrapper problem or a timing problem?" It's a design choice that narrows things to a single variable when something breaks.

Why write the semantic diff check in Python

Run XML through a text diff and you get piles of line differences from nothing but attribute order and whitespace. Eyeballing "did any key I didn't want to change get changed?" falls apart at 44 files. The right approach is a key-by-key comparison with plistlib (Python standard library).

import plistlib
from pathlib import Path

def verify_minimal_change(original: Path, modified: Path) -> bool:
    orig = plistlib.loads(original.read_bytes())
    mod  = plistlib.loads(modified.read_bytes())

    # ProgramArguments以外のキーが変わっていないか
    for key in orig:
        if key == "ProgramArguments":
            continue
        assert orig[key] == mod.get(key), f"意図しないキー変更: {key}"

    # 既存コマンドのsuffixが保持されているか
    orig_args = orig["ProgramArguments"]
    new_args  = mod["ProgramArguments"]
    assert new_args[-len(orig_args):] == orig_args, \
        "既存ProgramArgumentsのsuffixが変わっている"

    return True

Run this verification across every target to mechanically confirm "only ProgramArguments changed, and the existing command suffix matches exactly" — only then proceed to plutil -lint → bootout → bootstrap.

The `bootout → bootstrap` retry pattern

Retirement (just stopping) is complete with bootout, but when a reload is needed, use this shape.

reload_job() {
  local label="$1"
  local plist_path="$2"
  local domain="gui/$UID"

  # bootout（No such process は正常・無視してよい）
  launchctl bootout "${domain}/${label}" 2>/dev/null || true

  # bootstrap は最大15回・1秒間隔でリトライ
  # カーネル側のサービス消滅完了を待つため
  local max_retry=15
  for i in $(seq 1 $max_retry); do
    if launchctl bootstrap "$domain" "$plist_path" 2>/dev/null; then
      break
    fi
    if [ "$i" -eq "$max_retry" ]; then
      echo "RELOAD FAILED: $label" >&2
      return 1
    fi
    sleep 1
  done

  # live定義で検証（listではなくprintを使う）
  if launchctl print "${domain}/${label}" | grep -q "quota-guard"; then
    echo "RELOAD OK: $label"
  else
    echo "RELOAD WARN: live定義にラッパが見えない $label" >&2
    return 1
  fi
}

The key to this pattern is its asymmetry. bootout errors can be ignored; bootstrap errors must not be. bootout is idempotent (an operation toward a state that's already "not there"), but if bootstrap fails, the job is left unloaded and gone. Swallow that with || true and you'll land in the next trap.

`launchctl print` is the only way to see the real state

# 退役確認（消えていればOK）
launchctl list | grep com.shun.zenn-daily
# → 出力が空であればアンロード済み

# 再ロード後の定義確認（新しいProgramArgumentsが反映されているか）
launchctl print gui/$UID/com.shun.zenn-daily | grep "program ="
# → program = /path/to/quota-guard.sh  であればラッパが効いている

The output of launchctl print contains a line program = with the path of the binary that actually gets launched. If this still shows the old claude binary, you're in the state of "I edited the file but the reload didn't take." Even if the string quota-guard shows up in the logs, unless you check this line, you've only convinced yourself it went through.

Where I got stuck

When managing 160+ jobs, three patterns overlap: "I stopped it but it's running," "I changed it but it didn't take," and "it's running but producing zero results." Here are four cases where I actually got stuck, in symptom → cause → fix order.

① A "retired" job quietly died the day after a folder move

Symptom: The three ai-portraits series jobs — blur, partial, and swim — all started failing together with 0/2 (zero targets) from the morning after I moved their working folder. The logs just showed exit 1 stacking up. From launchd's point of view it was only "the job ran (and failed)," so nothing got unloaded.

Cause: I'd assumed the .plist.retired rename alone had stopped them, and never ran launchctl bootout. The morning after the rename on a long-running Mac, no session boundary had been crossed, so the jobs were still alive. Since I'd already moved the working folder the day before, all three jobs immediately hit exit 1 with "folder missing."

Fix: Just keep the order — get bootout through before moving the folder.

# フォルダを移動する前に必ずやる
for label in com.lily.ai-portraits-blur com.lily.ai-portraits-partial com.lily.ai-portraits-swim; do
  launchctl bootout gui/$UID/$label 2>/dev/null || true
done

# テーブルから消えたか全件確認
launchctl list | grep ai-portraits
# 出力が空になってからフォルダを移動する

# これが安全な順序
mv ~/dev/ai-portraits/work ~/dev/ai-portraits/work-archived

I had the order backwards. I pulled out the infrastructure prerequisites before declaring the intent to retire.

② `bootstrap` returned `5: I/O error` and erased the job

Symptom: After a rewiring, com.lily.threadspilot.engage had vanished from launchctl list. No firing traces in the script logs either. It was left unloaded and gone.

Cause: A race condition from calling bootstrap immediately after bootout on a running job. bootstrap ran before the kernel had finished tearing down the service, and returned 5: Input/output error. The reload script swallowed that error with 2>/dev/null and moved on to the next job as if it had succeeded. The result: only threadspilot.engage was left unloaded, with its registration gone.

Fix: Add the retry pattern shown above. Pair retries (up to 15 times at 1-second intervals) with a live-definition check via launchctl print. You can only say "bootout succeeded and bootstrap succeeded" once you've actually confirmed the program path in launchctl print. Discard errors only on the bootout side.

③ I trusted `launchctl list` and declared all 44 jobs "done"

Symptom: The day after bulk-inserting the quota guard into 44 jobs, I discovered some jobs were still calling the bare claude binary directly, bypassing the guard entirely. Quota usage was above expectations, and I only noticed after investigating.

Cause: The reload script confirmed "registered" via the output of launchctl list | grep $label and then printed RELOAD OK. But showing up in list is evidence of "registered," not evidence of "running with the new plist definition." Several jobs had failed bootstrap with 5: I/O error, and their definitions from the old session were still there, still showing in list.

Fix: Change the verification command from list to print.

# NG: listに出ることしか確認できない（旧い定義のままでも出る）
launchctl list | grep com.lily.some-job

# OK: live定義のprogramを確認する
launchctl print gui/$UID/com.lily.some-job \
  | grep "^  program =" \
  | grep -q "quota-guard" && echo "OK" || echo "NG: ラッパが入っていない"

I rewrote the script to run print on all 44 jobs and mechanically check whether the wrapper appears in program =, then re-ran bootout → bootstrap on the 6 jobs that still had old definitions.

④ A script was unconditionally overwriting the `CLAUDE_BIN` environment variable

Symptom: The ig-autoreply-ig-2 logs were full of "DM判定失敗: claude exit 1". The quota guard was supposed to be passing CLAUDE_BIN=~/.claude/scripts/quota-guard.sh to child processes, yet what was actually being called was the bare ~/.local/bin/claude.

Cause: Opening ~/dev/social-autolike/scripts/run-ig-autoreply.sh, the top of the file said this:

export CLAUDE_BIN=~/.local/bin/claude

An unconditional overwrite. Even if the launchd plist passes the wrapper path, the moment the child shell executes this line, the guard's path is gone. The same pattern existed in run-comment-reply.sh, run-rewrite.sh, and run-editor.sh. Four scripts in total. Wiring doesn't end at the plist. You have to check whether anyone along the path is overwriting unconditionally.

Fix: Change all four to the :- form.

# 変更前（無条件上書き・ガードを殺す）
export CLAUDE_BIN=~/.local/bin/claude

# 変更後（既存値がある場合は尊重する）
export CLAUDE_BIN="${CLAUDE_BIN:-~/.local/bin/claude}"

After the fix, I added a test case that asserts, with an exact match, that "this assignment line is written in the :- form." A test that only checks "does the string CLAUDE_BIN exist?" passes even when an old commented-out line is present. The existing test was exactly like that, and I missed it once. Instead of confirming existence with grep, you need to verify the form of the assignment line itself.

The structure of accumulating silent failures

What the four cases have in common is a structure where "the error is invisible."

Rename to .retired and forget bootout → the job keeps running, but nothing appears in launchd's error logs
bootstrap fails with 5: I/O error → if the script discards it with 2>/dev/null, silence
launchctl list shows jobs with old definitions as "registered" → looks normal on the surface, stale on the inside
Environment variable overwrite → you think the guard is in place, but it's bypassed. The job itself runs, so the exit code is normal

At a scale of 160 jobs, "running" and "running as intended" are different states. Feeling reassured because you see a log of the job running means your verification isn't deep enough.

To detect this problem regularly, I run the following inventory locally once a month.

# launchdに登録されている自分のジョブ一覧
launchctl list | grep 'com\.lily\|com\.shun' | awk '{print $3}' | sort > /tmp/loaded.txt

# LaunchAgentsにplistとして存在するジョブのLabel一覧
for f in ~/Library/LaunchAgents/com.{lily,shun}.*.plist; do
  defaults read "$f" Label 2>/dev/null
done | sort > /tmp/files.txt

# 差分を見る
echo "=== ファイルがないのにloaded ===" && comm -23 /tmp/loaded.txt /tmp/files.txt
echo "=== plistがあるのにunloaded ===" && comm -13 /tmp/loaded.txt /tmp/files.txt

If a line shows up under "loaded but no file," that's a ghost job. Jobs you thought you'd stopped with a .retired rename back in the day, still surviving because no session boundary was crossed, appear here.

com.shun.zenn-daily.plist.retired also shows up instantly if you run this inventory. Because the file's extension isn't .plist, it won't appear in /tmp/files.txt, but if bootout was never run, it will appear in /tmp/loaded.txt. That gap reveals the existence of a ghost job.

Changing a file's name and removing an entry from launchd's table are separate operations. If you have a mechanism that makes that gap visible, any retirement you forgot to finish will be caught in next month's inventory, without fail.

Stumbling points

The four cases where I got stuck are above. Here I'll list, comprehensively, the points that are "easy to do but rarely said out loud." Only things I actually hit while running a 160+ job fleet.

Skipped plutil -lint before bootout → bootstrap. If the plist has a syntax error, bootstrap looks successful and the job even shows in launchctl list, but it never fires — not once. macOS silently ignores broken plists. Even in launchctl print it stays at state = waiting and doesn't run when the time comes. Twice I spent an hour investigating with no idea of the cause, only to finally run plutil -lint com.lily.something.plist and see the syntax error. Whenever you touch a file, always go plutil -lint → bootout → bootstrap → launchctl print, in that order. plutil -lint alone comes before every other step.
Wrote a tilde (~) in a path inside the plist. launchd is not a shell, so it does not expand ~ to /Users/yourname. This is the cause of the vast majority of "the job fires but keeps ending with exit 127: command not found." If you look at the pre-change ProgramArguments in ~/Library/LaunchAgents/com.shun.zenn-daily.plist.retired, it doesn't actually say ~/.discord/run-and-notify.sh. cat the real file and you'll find absolute paths. Everything inside a plist is an absolute path. Be strict about the split: tilde notation in articles and explanations, absolute paths in the real file.
Forgot to include the three keys ProcessType, Nice, and LowPriorityIO in a new plist. These three keys are what let macOS lower a job's priority.

  <key>ProcessType</key>
  <string>Background</string>
  <key>Nice</key>
  <integer>10</integer>
  <key>LowPriorityIO</key>
  <true/>

Omit them and a job that's supposed to run in the background executes at the same priority as the foreground. At 160-job scale, the Mac gets noticeably sluggish at daytime peak. When I wired browser-slot into 23 jobs in August 2026, I found these three keys missing from 18 of them. Always include them in your template when writing a new plist.

Left StartCalendarInterval as a single dict and placed the firing time during hours when the Mac sleeps. If the Mac is asleep at a StartCalendarInterval firing time, macOS drops that firing entirely. cron catches up on accumulated runs at startup; launchd does not. If you placed a once-a-day job in the middle of the night, on days when the screen isn't on it may never run at all. Convert single-time jobs to a multi-slot dict array. The reason com.shun.zenn-daily has 4 slots — 7:30, 10:00, 14:00, 19:00 — is so that if one slot is lost to sleep, the others pick it up.
Switched to multiple slots without adding an idempotency guard, and posted 4 times in one day. Multi-slot is a "any one getting through is enough" design. Unless you simultaneously add a mechanism to skip the rest once the first slot succeeds (e.g., checking a state file for the day), all 4 slots go through and it runs 4 times. Changing StartCalendarInterval from single to multiple and adding an idempotency guard are a set. Do only one and you get real damage.
Designed the schedule interval based on "how long one job takes on its own." The tiktok-autopost case is a textbook example. I'd set the comment job to a 15-minute interval, but the job for another account sharing the exclusive lock (tiktok-avd.lock) uses the same lock. Fire a job that takes about 10 minutes per run at 15-minute intervals across 2 accounts combined (meaning the other one arrives every 7.5 minutes), and the lock is permanently occupied. Measured from July 21 to August 14, 2026: renappi launched 617 times with 255 exit 1s, bokuwalily launched 784 times with 458 exit 1s — all with zero results, while the follow lane hadn't updated its log in 3 weeks. Calculate intervals from the total occupancy of every job sharing the exclusive resource. A schedule designed around individual jobs' convenience will inevitably crush other lanes as scale grows.
Didn't verify a zero-result job's reason for existing before lowering its frequency. In the tiktok case above, I'd run the comment job 600+ times with the account still configured at commentActions=0. That a job is running (exit code 0) and that a job is producing its intended result are two different things. Before adjusting frequency, check "has this job produced even a single result today?"
Didn't set EnvironmentVariables, so PATH was missing. The default PATH for jobs launched by launchd is roughly /usr/bin:/bin:/usr/sbin:/sbin. Homebrew binaries (/opt/homebrew/bin) and nvm's Node (~/.nvm/versions/node/...) aren't in it. If you get command not found but the same command works when run manually from the terminal, this is it. Always add this to the plist:

  <key>EnvironmentVariables</key>
  <dict>
    <key>PATH</key>
    <string>/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
  </dict>

Used system/ as the domain for launchctl bootout. User-level LaunchAgents live in the gui/$UID domain. system/ is the LaunchDaemons domain, managed by root. Running bootout against system/com.shun.zenn-daily just ends with No such process and the job doesn't stop. A plist placed in ~/Library/LaunchAgents/ always uses gui/$UID.
Passed a relative path to the wrapper script's --log argument. Wrappers like run-with-retry.sh may cd internally. If you pass a relative path to --log, the wrapper expands it after changing directory, so the log gets written somewhere unexpected — or the file isn't created and the output is discarded. Always pass --log an absolute path.
Left a new job at RunAtLoad: true while also setting StartCalendarInterval. RunAtLoad: true also runs once the moment the plist is loaded (right after bootstrap). Combined with StartCalendarInterval, you get one run right after load plus one at each scheduled time. A case had crept into the fleet where RunAtLoad was temporarily set to true for debugging and then left there. RunAtLoad in a production plist is false as a rule.

Best practices

From the experience of maintaining a 160-job fleet, here are only the rules that, had I followed them, would have prevented an actual incident.

1. Whenever you touch a plist, always follow the order plutil -lint → bootout → bootstrap → launchctl print.
Break this order and you can no longer isolate why something doesn't work. plutil -lint is the one verification you can run "before stopping the job." Skip it and a broken plist silently surfaces.

2. bootout errors can be ignored; bootstrap errors must not be.
bootout is idempotent. Running it on an unloaded job just returns No such process, which isn't an anomaly. On the other hand, if bootstrap fails with 5: Input/output error, the job is left unloaded and gone. The only place 2>/dev/null belongs in a reload script is on the bootout side.

3. For verification, use launchctl print, not launchctl list.

# 退役確認（消えていればOK）
launchctl list | grep com.shun.zenn-daily

# 再ロード後の定義確認（新しい定義で動いているか）
launchctl print gui/$UID/com.shun.zenn-daily | grep "program ="

Showing up in launchctl list is evidence of "registered," not evidence of "loaded with the latest plist definition." Use launchctl print to verify the definition.

4. Build the three keys ProcessType=Background / Nice=10 / LowPriorityIO=true into your template.
Adding them by hand every time you write a new plist leads to omissions. Put these three keys in your standard template file and copy from it.

5. Make StartCalendarInterval a multi-slot dict array, not a single dict.
It's insurance against firings lost to sleep. With 3 slots, if one is lost to sleep, one of the remaining two gets through. When switching to multi-slot, always add an idempotency guard (skip if already succeeded today) at the same time.

6. Design schedules from the total occupancy of the exclusive resource.
If multiple jobs use the same lock, first compute the maximum frequency from the fleet-wide concurrency and per-run duration. Fitting 11 lanes into 120 minutes means the maximum adjacent interval is 120/11 ≈ 10.9 minutes. "All gaps 11+ minutes" and "strictly every 2 hours" are mathematically incompatible. If you compromise in the design, leave a comment stating explicitly which one you broke.

7. Write every path inside a plist as an absolute path.
~ isn't expanded because launchd doesn't run as a shell. Tilde notation is fine in articles and explanations, but the real file gets absolute paths only. The same applies when setting PATH in EnvironmentVariables.

8. Write environment variable assignments in the :- form.

# NG: ラッパが渡した値を上書きする
export CLAUDE_BIN=~/.local/bin/claude

# OK: 既存値がある場合は尊重する
export CLAUDE_BIN="${CLAUDE_BIN:-~/.local/bin/claude}"

Even with the wrapper wired in, if a shell script along the path assigns unconditionally, the guard never arrives. After wiring, sweep everything with grep -r 'CLAUDE_BIN' ~/dev ~/.claude/scripts and eliminate any assignment not in the :- form.

9. Put a count assertion on the test's LABELS constant.
Manage "the list of jobs that should have this wrapper" in a LABELS constant in the test, and assert the count too, like assert len(labels) == 44. When you add a job, the count test fails and "the new job doesn't have the wrapper" is detected mechanically. A test that only checks label existence passes even on an old commented-out definition.

10. Back up to a dated directory and confirm full byte-for-byte match with cmp before moving.

BACKUP=~/content/launchagents-backup-$(date +%Y%m%d)
mkdir -p "$BACKUP"
cp ~/Library/LaunchAgents/com.shun.zenn-daily.plist "$BACKUP/"
cmp ~/Library/LaunchAgents/com.shun.zenn-daily.plist "$BACKUP/com.shun.zenn-daily.plist" \
  && echo "backup OK" || echo "backup FAILED"

cp occasionally produces a file with a different byte count without raising an error (iCloud write delays, etc.). I make it a habit to confirm with cmp before touching the original.

11. Run a monthly inventory script on a schedule to flush out "ghost jobs."

launchctl list | grep 'com\.lily\|com\.shun' \
  | awk '{print $3}' | sort > /tmp/loaded.txt

for f in ~/Library/LaunchAgents/com.{lily,shun}.*.plist; do
  defaults read "$f" Label 2>/dev/null
done | sort > /tmp/files.txt

echo "=== ファイルなし・ロード済み（幽霊）===" && comm -23 /tmp/loaded.txt /tmp/files.txt
echo "=== plistあり・アンロード済み ===" && comm -13 /tmp/loaded.txt /tmp/files.txt

Lines under "no file, loaded" are ghost jobs. Any job you thought you'd stopped with a .retired rename that's survived across sessions will show up here without fail. Run it once a month and any missed step gets caught in next month's inventory.

12. Before lowering a zero-result job's frequency, verify its reason for existing.
That a job is running (exit 0) and that it's producing results (posts, follows, replies actually happening) are different things. There was a real case where, past the point of judging "it's running" from exit 0 in the logs alone, everything had been zero for 3 weeks. Frequency adjustment comes later; first, check in the actual logs whether "this job produced even one real result today."

Summary

com.shun.zenn-daily.plist.retired still exists on the filesystem today. The file name changed, but the Label inside is still com.shun.zenn-daily. Unless launchctl bootout gui/$UID/com.shun.zenn-daily was run, this job fires again today at 7:30, 10:00, 14:00, and 19:00.

Declaring retirement and executing it are separate operations.

The rename is the declaration. It doesn't change launchd's table.
bootout is the execution. Only once this goes through does the entry leave the table.
launchctl print is the verification. Only after confirming the live definition with print, not list, is retirement "complete."

As automation scales up, you reboot the Mac less often. The longer a session goes without a reboot, the longer jobs you "stopped" with a rename alone survive. On a fleet past 160 jobs, "I stopped it but it's running," "I changed it but it didn't take," and "it's running but producing zero" all happen at the same time.

One monthly inventory script and the bootout → list → print verification habit prevent almost every failure in this class. The commands are three lines. It's not about the procedure — it's entirely about whether you know.

How many jobs are sitting in your launchctl list right now that you're sure you retired?

The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day procedure are in a paid note (Japanese).
📕 Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

3 Duplicate Instagram Posts in 8 Hours: Anatomy of a False-Negative Success Check

Lily — Thu, 17 Sep 2026 05:00:06 +0000

I went from ¥100k/month as a student, to ¥600k juggling side gigs, to zero after a layoff, and then spent six months building an autonomous Claude Code environment that brought me back to ¥1.2M/month. What I learned along the way wasn't a string of automation wins. It was a pile of lessons about what happens when automation breaks.

On August 28, 2026, the same Instagram carousel showed up three times on my profile. Shortcodes DckjvpPoFRt, DckN-yYkIBsJ, and DclaPYooPjB — posted at 13:58, 17:55, and 21:54 JST. All three times, the job logged "post not found on profile." All three times, a direct check against the Instagram API showed the post had gone through.

The operation succeeded, and the check called it a failure — a false negative that repeated a side effect three times over.

Why this setup works at all

"Building an environment" means driving the number of times I have to touch anything toward zero.

For Instagram, launchd fires a script on schedule, and that script runs caption generation, posting, landing verification, and ledger write-back end to end with no human in the loop. Around 171 jobs run side by side in total, and the Instagram carousel post is one lane. I never have to go check "did today's post go out?" — the ok:true entry in the ledger and the Discord notification tell me instead.

The reason this is an "environment" rather than "work" is that the environment also decides when to retry.

If landing verification (_verify_landed()) returns ok:false, the same job runs again at the next scheduled slot. Instead of a human going "oh, that failed, let me try again," the scheduler re-runs it automatically. Zero time, zero effort — as long as the verdict is correct.

When the verdict is wrong, the mechanism runs in reverse. If the post succeeded but ok:false keeps coming back, the scheduler interprets that as "still not posted" and sends the same content again. The side effect (the post) is already done, but the verification layer insists it isn't. Let retries run in that state and identical posts pile up without limit.

"Producing more" is worse than "stopping"

A false positive — "it actually failed, but we called it a success" — stops the shipping line. That's a visible failure. Someone notices and investigates.

A false negative — "it actually succeeded, but we called it a failure" — increases shipments. That's an invisible failure. The ledger fills with ok:false, and on the surface it looks like "posts are running late." Meanwhile the same post is being mass-produced, and the script calmly schedules the next retry.

For an operation with side effects, the damage from a false negative is not "a failed check" but "a duplicated side effect." Every wrong verdict leaves another trace in the real world. In Instagram's case, that trace was three identical carousels lined up on my profile.

My learning notes record it this way: "[[false-positive-stops-the-line]] was the 'a false alarm halts shipping' pattern. This one is a false alarm that increases shipping. Worse than stopping."

The condition under which an environment works

171 jobs run efficiently because each one can accurately judge whether its own work is done. If the verdict is accurate, a successful job runs once, and only failed jobs get re-executed.

When the verdict breaks, that premise collapses. Successful jobs get treated as failures and re-run too. If the side effect is idempotent (same result no matter how many times you do it), no problem — but a social media post is not idempotent. Send once, one post goes public. Send three times, three go public.

Before asking "how many retries should this have?", you have to ask "is this operation idempotent?" Attaching three retries to a non-idempotent operation is a declaration that you'll tolerate the side effect up to three times. My retry-design reference (retry-and-giveup-design.md) says "throwing the same failure three times doesn't change the result," and it's the same point — when the problem is in the input to the verdict itself, adding attempts changes nothing.

The full sequence

Here's the timeline of what happened on the day of the incident.

[launchd: 13:58 JST]
        │
        ▼
  ig_autopost.py
        │
        ├─ (1) キャプション生成・投稿実行
        │       IG カルーセル送信 ─────────────► 成功 ✓ (DckjvpPoFRt)
        │
        └─ (2) 着地確認: _verify_landed()
                │
                ├─ プロフィール DOM 取得
                │   キャプション先頭12文字: "👾 個人開発の量産..."
                │   IG レンダリング:         <img alt="👾"> ← inner_text に出ない
                │   照合文字列(DOM側):        " 個人開発の量産 v"  ← ズレる
                │
                └─ 照合NG → ok: false → 台帳: 「投稿されていない」
                        │
                        ▼
[launchd: 17:55 JST] ← スケジューラ「まだ未投稿」と判断・再実行
        │
        ├─ 同じキャプションで投稿 ──────────────► 重複1本目 (DckN-yYkIBsJ)
        └─ _verify_landed() → ok: false ← また偽陰性
                        │
                        ▼
[launchd: 21:54 JST] ← さらに再実行
        ├─ 投稿 ───────────────────────────────► 重複2本目 (DclaPYooPjB)
        └─ _verify_landed() → ok: true ← ようやく成功判定

Three posts in eight hours, all the same carousel. The script exited all three times believing it had "worked correctly." Only checking the IG API revealed the duplicates.

Root cause: the matching input was broken

_verify_landed() fetched the profile page DOM and compared the first 12 characters of the body text with the first 12 characters of the caption. The design assumed "the DOM's inner_text contains the entire caption."

But Instagram renders emoji as <img> tags. When the caption starts with 👾, the DOM contains <img alt="👾">, and that character never appears in inner_text.

キャプション文字列: "👾 個人開発の量産 vol..."
照合しようとした文字列(先頭12文字): "👾 個人開発の量産 vo"
DOM inner_text(先頭12文字):         " 個人開発の量産 vol."  ← 👾 が img に化けて消える

→ 永久に不一致

This kind of false negative doesn't break on "did the click work (did the post go out)?" — it breaks on "what did you compare?" The matching mechanism runs fine. The material it's matching is corrupted.

The fix, in three parts

1. Caption normalization (strip emoji and full-width characters)

Transform the match key into a form that doesn't depend on how the DOM renders. By passing both sides — the caption string and the body text pulled from the DOM — through the same function, neither side can drop something the other keeps.

def normalize_caption_key(text: str) -> str:
    import unicodedata
    # 全角・半角空白を除去
    text = "".join(c for c in text if not c.isspace())
    # BMP外（絵文字 ord > 0xFFFF）と Unicode カテゴリ So/Sk を除去
    text = "".join(
        c for c in text
        if ord(c) <= 0xFFFF
        and unicodedata.category(c) not in ("So", "Sk")
    )
    # 先頭24文字（残りが24文字未満ならキャプション全体）
    return text[:24] if len(text) >= 24 else text

The match width grew from 12 characters to 24 as a margin for the emoji that normalization strips out. The original 12-character rule was designed for unnormalized strings; for the normalized remainder to carry the same amount of information, it has to be longer.

"Apply the same normalization to both the caption and the page body" — that's the crux. Normalize only one side and the transformed lengths won't line up, producing a different mismatch.

2. API-first lookup, with the DOM fallback cut off

The DOM's inner_text depends on how IG renders things, and IG can change that at any moment. An API-first lookup removes that uncertainty.

Using the ds_user_id cookie from the logged-in page, the script queries the internal API directly. If the API returns JSON (i.e., it responded normally) but the post isn't there, the result is locked in as a failure without falling back to the DOM. Only when all 8 queries return non-JSON (the API is broken, etc.) does it fall back to a single DOM check.

API 問い合わせ (最大8回)
        │
        ├─ JSON 返答あり + 投稿あり  → ok: true
        ├─ JSON 返答あり + 投稿なし  → ok: false（DOM に逃がさない）
        └─ 8回とも JSON 以外         → DOM を1回だけ確認 → ok / false

This is designed so that "couldn't verify" is never mistranslated as "not posted." If you escape to the DOM whenever the API is down, you're right back to inner_text matching, and the false-negative path stays open.

3. Fail-closed design

When API verification is impossible, treat the default as "already posted." The reasoning is asymmetry.

"Misjudged as not posted → re-run → duplicate": the actual damage this time (3 duplicates, cleanup work afterward)
"Misjudged as posted → skip → one slot's delay": naturally recovered at the next scheduled run

A duplicate can be deleted later, but it was public for a while and someone has to delete it. A delay is absorbed by the next job. For asymmetric damage, fail-closed — defaulting to the lighter outcome — is the right call.

Alongside this, I introduced an already-live verdict so that posts that are already public don't get recorded as failures in the ledger.

# 既存投稿として検出した場合
return {"ok": True, "note": "already-live"}

If it lands in the ledger as ok:false, the next retry check treats it as "not posted." Recording ok:true is what makes the duplicate guard actually work.

All three fixes landed without touching IG at all — static implementation changes plus pure-function tests (pytest 12 passed). The pre-post duplicate check (has_existing_caption()) was also wired in as a shared function for both the carousel and reel lanes, and it likewise stops fail-closed when the API can't be verified.

Implementation details

`normalize_caption_key` — why it ended up this way

Let's break the normalization function from above down one more level.

def normalize_caption_key(text: str) -> str:
    import unicodedata
    text = "".join(c for c in text if not c.isspace())
    text = "".join(
        c for c in text
        if ord(c) <= 0xFFFF
        and unicodedata.category(c) not in ("So", "Sk")
    )
    return text[:24] if len(text) >= 24 else text

The condition ord(c) <= 0xFFFF keeps only characters within the BMP (Basic Multilingual Plane). Most emoji sit at U+1F000 and above, meaning ord exceeds 65535. 👾 has an ord of 128126, so it's excluded immediately. The Unicode categories So (Symbol, Other) and Sk (Symbol, Modifier) are also removed because decorative symbols exist inside the BMP too. ★ (U+2605, category So), for instance, fits in the BMP but can't be relied on to appear consistently in inner_text.

The 24-character match width was worked backwards from measurement. The original 12 characters were designed for unnormalized strings. After normalization, the string is shorter by however many emoji were dropped. With a caption opening like 👾 個人開発の量産 vol.XX ——, the first 12 characters after 👾 falls away become 「個人開発の量産 v」 — a fairly generic string. Extending to 24 reads through 「個人開発の量産 vol.18 ——」, which is much less likely to collide with another post.

The key point is what the function is applied to. The same normalize_caption_key goes over both the caption string and the page body pulled from the DOM. Normalize only one side and the post-transform character counts diverge, producing a fresh mismatch. What pytest verified was exactly this scenario: "do both sides match after going through the function?"

# テストの骨格
def test_normalize_both_sides():
    caption = "👾 個人開発の量産 vol.18 —— "
    dom_text = " 個人開発の量産 vol.18 —— "  # IG が img に変換した後
    assert normalize_caption_key(caption) == normalize_caption_key(dom_text)

It's a pure function, so it can be verified without connecting to IG at all. That's what "12 passed in pytest with no IG connection" actually means.

API-first lookup — separating "couldn't verify" from "not posted"

Turning the flow above into code, the decision branches look like this:

def _verify_via_api(shortcode: str, caption: str) -> dict:
    for attempt in range(8):
        resp = _fetch_profile_api(user_id=DS_USER_ID, cursor=...)
        if resp is None:  # JSON以外が返ってきた
            continue
        # JSONが返った = API正常
        for post in resp.get("edges", []):
            node_text = post["node"].get("edge_media_to_caption", {})
            key = normalize_caption_key(node_text.get("edges", [{}])[0].get("node", {}).get("text", ""))
            if key and key == normalize_caption_key(caption):
                return {"ok": True, "source": "api"}
        # JSONが返ったが該当なし → DOMへ逃がさない
        return {"ok": False, "source": "api", "reason": "not_found"}
    # 8回ともJSONでなかった → DOMを1回だけ
    return _verify_via_dom(caption)

The core is the asymmetry between the resp is None branch and the return {"ok": False} branch. If "the API is up and returned JSON" but "the post isn't found," that is definitive information: it wasn't posted. There's no reason to escape to the DOM. Only when "none of the 8 attempts returned JSON" does it fall back to a DOM check.

Why 8? Requests to the internal API sometimes hit Instagram's rate limiting and return normally after one or two failures. But if all 8 come back non-JSON, either the API is broken or the session has expired, and asking further won't change the outcome.

The reason for using the ds_user_id cookie is that the logged-in internal API returns post lists as JSON even for profiles you don't follow. Pagination uses the after cursor when needed, but landing verification only checks the most recent posts, so one page is enough.

Fail-closed and the `already-live` ledger design

# API確認が8回とも不能 → DOMへ行ったが結果も不明 → fail-closed
def _verify_landed(shortcode: str, caption: str) -> dict:
    result = _verify_via_api(shortcode, caption)
    if result.get("source") == "api":
        return result
    # DOMフォールバックの結果
    dom_result = result
    if dom_result.get("ok") is None:  # 確認不能
        return {"ok": True, "note": "unverified-assumed-live"}
    return dom_result

The record format ok: True, note: "already-live" exists because the ledger reader uses ok:true for its skip decision. Recorded as ok:false, the entry gets reprocessed as "not posted" at the next retry. The note field is there so a human reading the ledger can see "why was this treated as success?" — the script logic doesn't use it. Without that distinction, three months from now you'll open the ledger and have no answer to "why is this one true?"

A second check based on post-count delta is included as well. It fetches the profile's post count before and after posting; if the difference is +1, it declares success independently of caption matching. This is a safety net for cases where caption normalization fails to match for whatever reason. Both checks use the same internal API, but they look at different "material" (caption vs. count), so if one breaks, the verdict doesn't die completely.

Where I got stuck

The day after the fix, 16 empty posts showed up

On August 30, two days after the August 28 fix (normalization + API-first + fail-closed), 16 carousels with no body text went public. Two patterns, eight rounds each.

The symptom was "duplicates again," but this time the cause wasn't matching — it was the posting side. keyboard.insert_text in ig_autopost.py never made it into the caption field, and the screenshots showed "Share" being pressed at 0/2200.

This incident was nastier than the previous one, as my learning notes put it:

🔴 The pre-post duplicate guard (has_existing_caption) and the post-post landing check (_verify_landed) share the same internal API and the same 24-character key (normalize_caption_key). With an empty body, neither matches → the duplicate guard says "not out yet," the landing check says "not out" → treated as failure, same ID re-posted.

Extracting the pre-post duplicate check into a shared function on August 28 was the right call. But as a result of sharing it, the failure mode was shared too. A single function is correct from a reuse standpoint, and it's fine as long as the caption goes in properly. But against the path where the caption comes out empty, every guard that shares it is neutralized simultaneously.

The fix came in two stages.

Stage one: confirm the caption went in before posting.

def insert_caption_with_verify(page, caption: str) -> None:
    # 手段1: keyboard.insert_text
    # 手段2: clipboard経由でpaste
    # 手段3: JSでvalue直接セット
    for method in [_keyboard_insert, _clipboard_paste, _js_set_value]:
        method(page, caption)
        actual = page.locator('[data-lexical-editor]').inner_text()
        if len(actual) >= len(caption) - 5:  # 5文字マージン
            return
    raise RuntimeError(f"キャプション投入失敗: 期待{len(caption)}文字/実測{len(actual)}文字")

"Did it go in?" is decided by the actual character count in the field, not by the return value of keyboard.insert_text. The API can report "inserted" while the DOM rendering hasn't caught up. In a live run, it printed キャプション投入 1257文字 / 期待1257文字 and then landing verification succeeded in 13 seconds.

Stage two: block the path where the match key becomes empty, at the entrance.

def normalize_caption_key(text: str) -> str:
    # ...（前述の正規化）
    result = text[:24] if len(text) >= 24 else text
    if not result:
        raise ValueError("照合キーが空: 投稿前に中止してください")
    return result

An empty key isn't "no match" — it's a third state: "the comparison never happened." Record the result of a comparison that never happened as ok:false in the ledger and a retry fires, multiplying the side effect. Stopping at the entrance with an explicit error breaks the chain of "can't compare → re-post."

TikTok threw errors for 15 straight days, but every post had gone through

On September 8, I discovered that post_reel_tiktok.py had been exiting with RuntimeError every time since September 24. For 15 days, the ledger accumulated a daily ok:false. But when I checked the TikTok dashboard, every single post had succeeded.

The cause: the success check depended on the vendor's fixed wording. After clicking Post, the script polled the page body for 120 seconds waiting for one of five fixed strings to appear.

SUCCESS_TEXTS = [
    "動画が投稿されました",
    "投稿を作成しました",
    "コンテンツが公開されました",
    # ...他2種
]

TikTok was navigating to the content management page after posting (URL containing /tiktokstudio/content), but the sidebar label on that screen had changed from 「管理する」 to 「管理」. Nothing in SUCCESS_TEXTS matched, so it threw RuntimeError every time.

There are two reasons it went unnoticed for 15 days. First, the ledger's skip logic only looked at ok:true entries. Video IDs recorded as ok:false became candidates for reprocessing as "not yet posted" on the next run. They had been posted, so had a re-run actually happened, the same video would have gone public twice. The only reason no duplicates appeared is that the ok:false IDs didn't happen to reach the front of the queue — new videos were prioritized.

Second, the failure exception carried neither the page URL nor the start of the body. A single line — RuntimeError: タイムアウト: 成功文言が見つかりません — gives you no way to diagnose "TikTok changed its wording." When an error leaves no evidence at the scene, recovery time balloons. The 15 days were caused by that missing record.

The fix was to layer the checks.

def _is_posted_successfully(page) -> bool:
    # 判定1: 従来の成功文言（TikTokが変えるかも）
    if any(t in page.inner_text("body") for t in SUCCESS_TEXTS):
        return True
    # 判定2: URLがuploadを離れてcontentに着いた
    if "/tiktokstudio/content" in page.url:
        return True
    # 判定3: 自分が投げたキャプションの先頭12文字が本文にある
    if normalize_caption_key(caption)[:12] in page.inner_text("body"):
        return True
    return False

Check 3 — "verify using the caption I submitted" — is the most robust. Even if the vendor changes the UI, if I'm the one who posted, my own text will be on the screen. Checks 1 and 2, which depend on the vendor's vocabulary, become fallbacks.

My learning notes record it like this:

A check that depends on vendor wording can't be fixed unless you preserve the evidence at the moment it fails

Designing your error records matters as much as designing your retries. Always attaching page.url and the first 200 characters of body to the exception dates from this incident.

X (formerly Twitter) quote posts showed "0" for five days straight

On September 10, I noticed that every quote post since September 5 had been counted as quoted=0. They had actually been posted, but the landing verification script returned unverified for all of them and never incremented the count.

The cause was Playwright's strict mode combined with a DOM structure change.

// 変更前
const tweetText = await article.locator('[data-testid="tweetText"]').innerText()

A quote post's article element contains two tweetText nodes: my own text and the embedded original post. In strict mode, Playwright's innerText() throws when multiple elements match. That error was caught and turned into an empty string, the comparison fell to unverified, and the count stayed at quoted=0.

The fix is .first() to read only the first element.

const tweetText = await article.locator('[data-testid="tweetText"]').first().innerText()

A one-line change, but it stopped five days of false alerts. health.mjs had been pushing "0 today" notifications to the desktop, so for five days the state was "all quote posts are failing, cause unknown." In reality nothing was broken and every post had succeeded.

This incident has exactly the same structure as TikTok's 15 straight days of exit 1. "When the vendor's DOM goes from one element to multiple, a strict comparison falls to failure and produces a false negative" — same structure, different platform, two weeks later. If I'd been checking that pattern across lanes after learning it once, it would have been closed in 0 days, not 5.

This is the one that stung most. I knew this failure mode, and still couldn't prevent its recurrence on another platform. After the fix, I preemptively added .first() to the reply.mjs in every lane using the same kind of comparison — but that was after the fact.

What the three failures have in common

The 16 empty-caption posts on August 30, TikTok's 15 consecutive exit 1s, X quote posts at zero for 5 days — what they share is this structure: the comparator can't tell you that the comparator itself is broken.

The script keeps returning ok:false. The ledger accumulates failures. The scheduler books re-runs. Nowhere in that chain is there a signal saying "the input to the verdict is broken." The script is working correctly — with the wrong material.

My retry-design reference (retry-and-giveup-design.md) has this line:

Throwing the same failure three times doesn't change the result. Suspect what you're passing in, not the count.

All three cases are exactly this. Cut retries to two, leave them at three — as long as the comparison material is broken, nothing changes. When the verdict falls the same way three times in a row — consecutive false negatives — the first thing to suspect is "what am I using as comparison material?"

inner_text's rendering dependency, a vendor changing fixed wording, strict mode's element-count sensitivity — all three are the pattern "the comparison logic I wrote is correct, but the outside environment invalidated the comparison." Adjusting retry counts is powerless against this pattern. You either change the material or change how you compare. There's no third option.

In my current environment, when landing verification fails, the next step is not an immediate retry but asking "why did it fail?" through a separate path. On failure, it hits the API directly to check "does the post exist?" If it does, it's recorded as already-live. Only if it doesn't is "genuinely not posted" confirmed. That one extra step — "verify, then decide" — is the last wall against duplicated side effects.

Pitfalls

A full list of the landmines I actually stepped on. Every one follows the pattern "the script exits normally, and only the real world is broken."

Symbols other than emoji also vanish from inner_text. ★ (U+2605, Unicode category So) is a BMP character, but it can't be relied on to appear consistently in inner_text under IG's rendering. I thought ord(c) <= 0xFFFF was enough; this is why categories So (Symbol, Other) and Sk (Symbol, Modifier) are additionally stripped. Base normalization on "is it in the BMP?" alone and the next incident is on its way.
Normalizing the match key on only one side. Normalize just the caption string and compare against the raw DOM text, and the character counts diverge by however many emoji were dropped, creating a new mismatch. Always pass normalize_caption_key over both sides — the easiest thing to forget in the whole implementation.
Not closing the path where the match key becomes empty. This was the root cause of the 16 body-less carousels on August 30. keyboard.insert_text in ig_autopost.py never reached the caption field, and "Share" was pressed at 0/2200. The pre-post duplicate guard (has_existing_caption) and the post-post landing check (_verify_landed) share the same 24-character key, so the moment the key goes empty, both are neutralized at once. The duplicate guard says "not out yet," the landing check says "not out," and the same content keeps getting retried.
"Prevention" and "detection" sharing the same comparator. Correct from a code-reuse perspective, but the instant that function breaks, prevention and detection die simultaneously. A hidden single point of failure. At least one of them needs to verify with different material (post-count delta, API count).
Success checks that depend on the vendor's fixed wording. TikTok navigated to a different page after posting and the sidebar label changed from 「管理する」 to 「管理」 — that alone broke all five registered strings. post_reel_tiktok.py threw RuntimeError on every post and stacked ok:false for 15 days. Only opening the TikTok dashboard directly revealed that everything had posted.
Failure exceptions with no page URL and no body prefix. One line — RuntimeError: タイムアウト: 成功文言が見つかりません — can't diagnose "TikTok changed its wording." Because page.url and the first 200 characters of body weren't in the exception, isolating the cause took 15 days.
Playwright strict mode throwing on multiple elements. An X (formerly Twitter) quote post's article element has two [data-testid="tweetText"] nodes — my own text and the embedded original. innerText() threw in strict mode, returned an empty string, the comparison fell to unverified, and every quote post since September 5 was counted as quoted=0. One added .first() fixed it, but it took five days to notice.
A design that mistranslates "couldn't verify" as "not posted." Fall back to the DOM when the API is down and you're back to inner_text matching. "The API returned JSON but the post isn't found" is definitive (not posted); "none of 8 attempts returned JSON" just means it couldn't be verified. Escape to the DOM without distinguishing the two and the false-negative recurrence path stays open.
Throwing the same failure three times with more attempts. note-autolike's rewrite produced zero output for three days because the validator rejected titles over 58 characters while the generation prompt said nothing about a length limit. askClaude simply re-sent the identical prompt three times on failure, and got the same 63-character title three times. A constraint has to be written in both the validator and the generation prompt to function.
Exiting 0 immediately on slot-acquisition failure. browser-slot.sh printed SKIP: global limit reached and exited the instant it couldn't grab one of the global slots (3), so 12 jobs in the 12:00 hour, 9 in the 09:00 hour, and 8 in the 11:00 hour vanished without ever starting. After adding a 600-second wait option and randomizing the retry interval to 15–45 seconds, misfires dropped to nearly zero. A fixed interval makes every job stampede at the same instant (thundering herd), so the random spread is mandatory.
A fallback that lands on the same wall. In gen_note_thumbs.py I added a three-tier fallback — real Chrome → bundled chromium → env — and all three tiers ended in the same 90-second rc=-9. Same symptom doesn't mean same cause. Before wiring in a fallback, measure once whether that fallback hits a different wall than the original.
IG's delete API returning 200 without deleting. Call the delete API with an expired session (sessionid) and it returns 200 with an HTML body. Trying to delete the three August 28 duplicates, not one was removed. Don't define success as "the API returned 200" — the exact same lesson as post verification.
Best-effort pass-through that only surfaces downstream. In IG reel crop-ratio selection, there was a branch that silently passed through when query_selector returned None. The label varied by day — 「オリジナル」, 「Original」, 「9:16」, 「元の写真」, 「元の比率」 — while the code's candidate list was fixed. 9:16 wasn't selected, "Next" was pressed at 1:1, and a video with the sides cropped off went public on the grid. Select/toggle operations need measured confirmation that the selection took before moving to the next step.
Trusting the note CTA paste result. On September 2, note.com changed its behavior to drop quote blocks. The script verified by matching a signature string at the end, so it judged the stripped articles as "not applied" and kept re-pasting — producing 59 duplicate CTAs and 31 missing signatures. It should have verified by reading back the published page, not the editor's "inserted" record.
Reproducing the same-shaped bug on another platform. I'd learned the structure "DOM goes from one element to multiple → strict comparison falls to failure → false negative" on TikTok. Two weeks later I hit the exact same structure on X. The result of having no habit of preemptively fixing same-shaped spots across all lanes at fix time. Checking a pattern learned in one place across the board would have closed it in 0 days, not 5.

Best practices

Here are the patterns actually in use in my environment now, distilled from the pitfalls above.

1. Always pass both sides of a match key through the same normalization function

def normalize_caption_key(text: str) -> str:
    import unicodedata
    text = "".join(c for c in text if not c.isspace())
    text = "".join(
        c for c in text
        if ord(c) <= 0xFFFF
        and unicodedata.category(c) not in ("So", "Sk")
    )
    return text[:24] if len(text) >= 24 else text

Run the same function over both the caption string and the DOM-extracted text. One side alone leaves the post-transform lengths out of sync and produces a new mismatch.

2. Turn an empty match key into an error at the entrance

result = text[:24] if len(text) >= 24 else text
if not result:
    raise ValueError("照合キーが空: 投稿前に中止してください")
return result

An empty key isn't "no match (ok:false)" — it's the third state, "the comparison never happened." Stack it as ok:false in the ledger and the next retry fires, multiplying side effects.

3. Verify side-effecting operations against the other side's actual state

def _verify_via_api(shortcode: str, caption: str) -> dict:
    for attempt in range(8):
        resp = _fetch_profile_api(user_id=DS_USER_ID, cursor=...)
        if resp is None:
            continue
        for post in resp.get("edges", []):
            key = normalize_caption_key(...)
            if key and key == normalize_caption_key(caption):
                return {"ok": True, "source": "api"}
        return {"ok": False, "source": "api", "reason": "not_found"}
    return _verify_via_dom(caption)

"The API returned JSON but the post isn't found" is definitive — don't escape to the DOM. Only when none of 8 attempts returns JSON, use the DOM exactly once. "Couldn't verify" and "not posted" are different states.

4. Fail closed — treat "unverifiable" as "already posted"

The damage from a duplicate (cleanup work, time spent public) and from a delay (absorbed by the next job) is asymmetric. When you can't verify, pick the lighter one and record {"ok": True, "note": "unverified-assumed-live"} in the ledger. Recording ok:true is what prevents the next retry.

5. Use different material for "prevention" and "detection"

If the pre-post duplicate guard and the post-post landing check share the same 24-character key, one breaking kills both. Run the post-count delta (difference in profile post count before and after) as an independent verification material. With caption and count moving independently, one breaking doesn't kill the verdict completely.

6. Judge caption insertion by the field's actual character count

def insert_caption_with_verify(page, caption: str) -> None:
    for method in [_keyboard_insert, _clipboard_paste, _js_set_value]:
        method(page, caption)
        actual = page.locator('[data-lexical-editor]').inner_text()
        if len(actual) >= len(caption) - 5:
            return
    raise RuntimeError(f"キャプション投入失敗: 期待{len(caption)}文字/実測{len(actual)}文字")

"It went in" is decided by the field's actual character count, not the API's return value. Try three methods and verify by reading back — in a live run it printed キャプション投入 1257文字 / 期待1257文字 and then passed landing verification in 13 seconds.

7. Judge success by the text you submitted, not vendor wording

def _is_posted_successfully(page, caption) -> bool:
    if any(t in page.inner_text("body") for t in SUCCESS_TEXTS):
        return True
    if "/tiktokstudio/content" in page.url:
        return True
    if normalize_caption_key(caption)[:12] in page.inner_text("body"):
        return True
    return False

Check 3 — "verify using the caption I submitted" — is the hardest to break. The vendor can change its UI, but the text I posted will be on the screen. Fixed strings are the fallback.

8. Always attach page.url and the first 200 characters of body to failure exceptions

When a vendor-wording-dependent check breaks, diagnosis takes 15 days if there's no evidence at the scene. Always put page.url and page.inner_text("body")[:200] in the exception's one line.

9. Use .first() explicitly under Playwright strict mode, and add it preemptively across all lanes

// 変更前: strict modeでthrow
const tweetText = await article.locator('[data-testid="tweetText"]').innerText()
// 変更後
const tweetText = await article.locator('[data-testid="tweetText"]').first().innerText()

At fix time, check every lane using the same kind of comparison and apply the same fix. This is the habit that prevents "learned it in one place, recurred on another platform."

10. When the same failure happens three times, suspect what you're passing in, not the count

Exactly as written in retry-and-giveup-design.md — "Throwing the same failure three times doesn't change the result. Suspect what you're passing in, not the count." Deterministic failures (validation violations, expired logins, vendor wording changes) must be classified before retrying, or adding attempts just burns resources.

11. Set retry counts by "how many times may this side effect occur?"

Attaching three retries to a non-idempotent operation declares that you'll tolerate the side effect up to three times. Before designing retries for social posts, ask "is this operation idempotent?" If not, always insert a step that checks the current state before retrying.

12. Wait up to 600 seconds for a slot instead of skipping immediately

BROWSER_SLOT_WAIT_SEC=600  # 0にすれば従来の即skip
# 再試行間隔はランダム（thundering herd 防止）
sleep $((RANDOM % 30 + 15))

BROWSER_SLOT_WAIT_SEC=0 exactly matches the old immediate-skip behavior, preserving backward compatibility with the 30+ existing launchd jobs.

13. Distinguish "waited / gave up / skipped" in the ledger log

If a record that gave up after waited=600s and a record that did nothing and hit exit 0 look like the same line, you can't tell "inefficient" from "never ran at all." To judge from the ledger whether a run of false negatives means "the comparator is broken" or "it genuinely isn't running," always log the reason.

14. When you add a retry, put fault injection in the same commit

# 本番では無効、注入時だけ有効
WA_API_FAIL_ONCE=2 node index.js
# → "api retry attempt=1/3" → "attempt=2/3" → exit 0 で完走
# 未設定なら retry ログが1行も出ない

Claiming you "added" a retry without fault injection proves nothing, because intermittent failures can't be reproduced and you can't show the path was actually exercised. Don't stop at showing the git diff.

15. Read back the published page before recording "applied"

Don't trust the editor's "inserted" return value. The 90 duplicate note CTAs happened because this verification step was missing. The principle reader-state-not-self-record — judge by the other side's actual state, not your own send log — applies to both post verification and paste verification.

Summary

In roughly two weeks starting August 28, three platforms broke the same way in succession. IG's 3 duplicates, TikTok's 15 consecutive exit 1s, X quote posts at zero for 5 days — every one has the structure "the comparator works fine, but the material it compares is broken."

The script throws no errors. The ledger keeps stacking ok:false, and the scheduler calmly books re-runs. Nowhere is there a signal saying "the job is broken." You only notice when you look at reality directly from outside — three carousels lined up on the profile, a post list checked on the dashboard, a ledger finally opened after a week of zero-count notifications.

One line from retry-and-giveup-design.md sums up this structure:

Throwing the same failure three times doesn't change the result. Suspect what you're passing in, not the count.

In my environment now, when landing verification fails, the next step isn't an immediate retry but asking "why did it fail?" through a separate path. Hit the API directly to confirm the post exists; if it does, record it as already-live with ok:true. Only if it doesn't is "genuinely not posted" confirmed.

That one extra step — verify, then decide — is the precondition for running 171 jobs autonomously. Building an environment means driving the number of times I have to touch anything toward zero. Keeping it at zero means continually asking what the verdict is actually based on. The comparator doesn't break. What breaks is the material the comparator is looking at.

One question for you: in your own automation, which success check is currently trusting a return value instead of reading back the other side's actual state?

The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day setup guide are in a paid note (Japanese).
📕 How I actually make money with an autonomous Claude Code environment — system, examples, getting started, support

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

--incremental Made My TypeScript Hook 3.6x Slower. A 200KB Threshold Fixed It

Lily — Thu, 17 Sep 2026 00:00:08 +0000

I started freelancing in college at ¥100k a month, stacked side gigs up to ¥600k, then got laid off and dropped to zero overnight. Over the next six months I built an autonomous Claude Code setup from scratch, and it now runs at ¥1.2M a month in revenue. Of everything that went into that, the bug that quietly ate the most time was the one where I tried to make something faster and made it slower instead.

Why This Setup Works

"Working" and "the environment is running" are two different things

When you're trying to grow revenue as a solo developer, the first wall you hit isn't how fast you write code — it's the lag in verification. If you catch a type error the instant you write it, fixing it takes a minute. If you catch it 30 seconds later, you have to reload the surrounding context from memory. At 60 seconds you're already drifting into the next task, so the context switch costs even more.

Claude Code has a mechanism that calls a PostToolUse hook after every tool execution. Put tsc --noEmit there and a type check runs every time the agent edits a file; if there's an error, feedback flows straight back into the next instruction. The structure closes the "write a file, then verify it" loop without a human in the middle.

The problem is that as the project grows, that "straight back" falls apart.

Why tsc's cold start is so heavy

The TypeScript compiler loads the entire dependency graph on startup. With 100 files it's snappy, but once imports chain out and the set of modules actually referenced balloons into the hundreds, re-parsing from zero every time is no small cost. When I measured it on a project called closet-os, a cold tsc --noEmit took 30–60 seconds. An environment where the hook blocks for 60 seconds on every tool call isn't autonomous — it's an obstacle that stops work.

The --incremental option exists to solve this. It does the full analysis only on the first run and caches the result in a .tsbuildinfo file. Subsequent runs re-analyze only the diff, so if the change is small it finishes in 1–3 seconds. For a hook, it looked like the ideal choice.

The case where it backfires

But when I added --incremental to the closet-os hook, cold runs got 3.6x slower.

The cause was the .tsbuildinfo file bloating. The bigger the project, the longer the initial cache generation for incremental diff data takes. On top of that, as the cache accumulates and the file size grows, the overhead of reading that giant JSON on every run stops being negligible. The "only process the diff" benefit was being outweighed by the "load a heavy cache every time" cost.

For small-to-medium projects, incremental wins overwhelmingly. But on large projects, plain --noEmit is actually faster — the relationship inverts. And since the hook reuses the same script across every project, I needed "a mechanism that switches automatically based on project size."

The threshold guard idea

The cache file's size correlates strongly with project size. Small .tsbuildinfo means incremental works; large means normal mode is faster. From that observation I arrived at a one-line guard: "if the cache file exceeds 200KB, fall back to normal mode."

The threshold was set from measurements. I confirmed both that the closet-os cache was around 200KB at the point where it started getting slow, and that projects smaller than that were consistently faster with incremental. It's not a magic number; it's a boundary based on real data.

The Overall Flow

The full picture of the hook

Claude Code がファイルを編集 (PostToolUse)
        │
        ▼
post_tsc_check.sh 起動
        │
        ├─ tsconfig.json が無い → exit 0（即終了）
        │
        ├─ timeout コマンド解決
        │    ├─ gtimeout (GNU coreutils) があれば優先
        │    └─ なければ timeout / フォールバック
        │
        ├─ TSBUILDINFO のサイズ確認
        │    ├─ ファイル無し or ≤200KB → --incremental モード
        │    └─ >200KB             → 通常モード（フォールバック）
        │
        ├─ tsc 実行（timeout 60s）
        │    ├─ --incremental: --noEmit --pretty false --incremental --tsBuildInfoFile
        │    └─ 通常:           --noEmit --pretty false
        │
        ├─ rc=124 (timeout) → 警告メッセージ・exit 0
        └─ エラーあり → 型エラー表示・exit 0（エージェントはログを受け取る）

The Claude Code agent can use the hook's output directly in its next reasoning step. When there are type errors they're printed with the prefix "=== TypeScript型エラー検出 ===" (TypeScript type errors detected), so the agent reads the errors itself and enters a fix loop.

The real code: from size check to branching

Let's read the core of ~/.claude/hooks/post_tsc_check.sh in order.

① Defining the cache path

CACHE_DIR="node_modules/.cache"
[ -d "node_modules" ] && mkdir -p "$CACHE_DIR" 2>/dev/null
TSBUILDINFO="$CACHE_DIR/tsc-hook.tsbuildinfo"

The .tsbuildinfo is pinned at node_modules/.cache/tsc-hook.tsbuildinfo. The reason it's not in the project root is that node_modules/ in .gitignore already covers it. The filename tsc-hook also keeps its namespace separate from the app's own build cache.

② The 200KB guard

USE_INCREMENTAL=1
if [ -f "$TSBUILDINFO" ]; then
  SIZE=$(stat -f %z "$TSBUILDINFO" 2>/dev/null || echo 0)
  if [ "${SIZE:-0}" -gt 204800 ]; then
    USE_INCREMENTAL=0
  fi
fi

stat -f %z is how you get a file's byte count with macOS's stat (the equivalent of Linux's stat -c %s). 2>/dev/null || echo 0 provides a zero fallback on error, so it runs safely on the first invocation when the file doesn't exist yet. 204800 is 200 × 1024, i.e. 200KB in bytes.

③ Building the arguments and running

if [ "$USE_INCREMENTAL" = 1 ]; then
  TSC_ARGS="--noEmit --pretty false --incremental --tsBuildInfoFile $TSBUILDINFO"
else
  TSC_ARGS="--noEmit --pretty false"
fi

if [ -n "$TIMEOUT_CMD" ]; then
  result=$($TIMEOUT_CMD npx tsc $TSC_ARGS 2>&1 | head -30)
  rc=$?
else
  result=$(npx tsc $TSC_ARGS 2>&1 | head -30)
  rc=$?
fi

--pretty false turns off color codes. The hook's output goes into the agent's log, not a terminal, so ANSI escape sequences mixed in would hurt readability. head -30 prevents an output explosion when errors pour out in bulk.

④ macOS support for timeout

TIMEOUT_CMD=""
if command -v gtimeout >/dev/null 2>&1; then
  TIMEOUT_CMD="gtimeout 60"
elif command -v timeout >/dev/null 2>&1; then
  TIMEOUT_CMD="timeout 60"
fi

macOS has no /usr/bin/timeout (on the BSD side, timeout comes as gtimeout from Homebrew's coreutils). We check for existence with command -v before assigning, and if neither exists, TIMEOUT_CMD="" stays empty and it runs without a timeout. Written in this order, the same script works on Linux, macOS, and CI.

⑤ Detecting a timeout

if [ "$rc" = 124 ]; then
  echo "=== TypeScript check timeout (60s exceeded — tsc 多重実行/巨大依存変更の疑い) ==="
  exit 0
fi

rc=124 is the exit code when timeout / gtimeout kills the process. It's treated as a warning rather than an error, and returning exit 0 doesn't block the agent's flow. The message mentions "suspected concurrent execution" because it actually happened: when Claude Code runs tools in parallel, multiple tsc processes start at the same time and corrupt each other's cache.

Why return exit 0

Even with type errors, the script ends with exit 0. With exit 1, the hook is treated as "failed," and there are cases where Claude Code's tool execution itself gets interrupted. Type errors are information for the agent, not a condition for halting the flow. If you write the error content to stdout, the agent reasons "there's a type error, so fix it" in its next step. It's a design that separates error detection from flow control.

Implementation Details (continued)

Why the tsconfig.json check is the first line

[ -f "tsconfig.json" ] || exit 0

The script begins with this one line. PostToolUse hooks fire in every project, so they trigger in repositories that aren't TypeScript at all. Putting the tsconfig.json existence check at the top means Python projects or Bash-only directories exit immediately with exit 0.

The key point is that this check runs in the direction of "exit if tsconfig.json is absent," not "exit on failure." When [ -f "tsconfig.json" ] is true, exit 0 doesn't run (the right side of || is evaluated only when the left side is false). If you're not used to reading this one-liner it'll confuse you later, so it's worth sorting out up front.

Designing the output block

if [ -n "$result" ]; then
  echo "=== TypeScript型エラー検出 ==="
  echo "$result"
  echo "================================"
fi
exit 0

The hook "prints nothing" when there are zero errors — that is, the normal case. If all types pass, stdout is empty and no noise lands in the agent's log. Only when there are errors does it print with the === TypeScript型エラー検出 === prefix, so the agent can determine whether type errors exist simply by checking whether that string appears in the output.

-n "$result" checks that the string is non-empty. With no type errors, tsc's output is an empty string, so this condition works as the equivalent of "type errors present = output present."

The 2>&1 | head -30 combination

result=$($TIMEOUT_CMD npx tsc $TSC_ARGS 2>&1 | head -30)

2>&1 handles the case where tsc writes errors to stderr. TypeScript errors normally go to stderr. Without merging them into stdout, $result gets nothing. head -30 limits output to the top 30 lines so the buffer doesn't overflow when errors are explosively numerous (cascading type errors producing hundreds of lines is not unusual).

Is there a basis for the number 30? There is. Looking at actual error logs from closet-os, the core of the error content (filename, line number, message) is concentrated in the first few lines. Thirty lines was enough for the agent to make a fix decision. Too many, and a single error consumes hundreds of tokens, degrading the hook's cost efficiency.

Why use `npx tsc`

I use npx tsc rather than calling ./node_modules/.bin/tsc directly. The reason is simple: even when node_modules doesn't exist (before the first npm install), npx falls back to a global tsc. The hook needs to run regardless of the project's setup state. When node_modules does exist, npx prefers the local one, so there's no version discrepancy.

Total line count of the script

The actual file is 61 lines. Excluding comments and blank lines, it's effectively under 40. "The clarity of 100 lines over 1,000" is something you can embody in places like this. The shorter the functional core, the easier it is to copy-paste when porting to another project later.

Where I Got Stuck

Everything below is a failure I actually hit. Symptom, cause, fix — in that order.

① Adding a pipe made `rc` always 0

Symptom: tsc is emitting type errors, but no error message ever comes back from the hook. Adding echo "$result" for debugging shows the output is captured correctly, yet rc stays 0.

Cause: The first implementation was written like this.

result=$(npx tsc $TSC_ARGS 2>&1 | head -30)
rc=$?

$? captures the exit code of the immediately preceding command. In a pipeline, $? returns the exit code of head. head -30 always succeeds (exit code 0), so no matter how many errors tsc emitted, rc=0 persisted.

Fix: I considered avoiding the pipe by capturing all output first and then running it through head, but that has buffer problems with massive errors. Currently the variables are two-tiered (result holds the head-filtered output, and rc looks at the exit code of the timeout command rather than immediately after the pipe). If you look at the actual script, rc=$? comes after the whole assignment, not after | head -30. As shell behavior goes, the exit code of a command substitution $(...) is that of the last command in the pipeline. timeout / gtimeout returns 124 only on timeout and otherwise passes tsc's exit code through transparently. In other words, by inserting timeout, the only thing inside the pipe is head, and timeout carries out tsc's code.

result=$($TIMEOUT_CMD npx tsc $TSC_ARGS 2>&1 | head -30)
rc=$?
# この rc は timeout コマンド全体の終了コード
# timeout が tsc を wrap しているので、tsc の rc が透過される
# (ただし timeout 自体の rc=124 がタイムアウトを示す)

I wasn't using timeout at first, which is how I fell into this trap. I only realized afterward that adding timeout had also become a side solution to the pipe problem.

② Getting the size failed with macOS `stat`

Symptom: I implemented the 200KB guard, but USE_INCREMENTAL=1 never changes regardless of state. The tsbuildinfo has clearly grown to several MB, yet no fallback.

Cause: I'd been reading Linux command references and wrote stat -c %s.

# 間違い（Linux 用）
SIZE=$(stat -c %s "$TSBUILDINFO" 2>/dev/null || echo 0)

macOS's stat is BSD-based and the options are completely different. -c %s is treated as an invalid option on macOS and returns an error. Since 2>/dev/null discards the error, echo 0 runs, and the size was always judged to be 0.

Fix: The macOS syntax is -f %z.

SIZE=$(stat -f %z "$TSBUILDINFO" 2>/dev/null || echo 0)

-f specifies a format string, and %z returns the file size in bytes. Same meaning as Linux's -c %s, different flag.

Discovering this required reading man stat and cost me 30 minutes. I've thought I should have just used wc -c < "$TSBUILDINFO" from the start, but wc -c reads the whole file, which is slightly slower on large files, and it needs extra error handling when the file doesn't exist. stat only reads filesystem metadata so it's faster, and the 2>/dev/null || echo 0 pattern handles absence safely in one line.

③ `exit 1` stopped the agent

Symptom: Every time the hook detects a type error and runs, Claude Code's tool execution loop gets interrupted midway. I can see errors are being emitted, but the loop ends before the agent starts reasoning "let me fix this."

Cause: The first implementation returned exit 1 on type errors. When a PostToolUse hook returns a non-zero exit code, Claude Code treats the tool execution as "failed," and there are cases where it doesn't continue subsequent steps.

Fix: I switched to a design that limits the hook's role to "information notification," not "flow control." Type errors are input information for the agent, not a reason to stop processing. By writing the error content to stdout and returning exit 0, the agent receives the information while continuing, and proceeds autonomously to the next action: "fix this type error."

Once I reframed it as "the hook is an information channel, not a control channel," the design cleaned right up.

④ ANSI escape codes polluted the agent's context

Symptom: The agent should be receiving the type error log, but it can't read the error content accurately and produces off-target fixes. When I paste the error message into my own terminal, mysterious strings like ^[[1m^[[31merror^[[0m are mixed in.

Cause: There was a period when I hadn't added --pretty false. TypeScript's default is colored output, which shows as red error text in a terminal, but the substance is ANSI escape sequences. When these get mixed into the hook's output, the text the agent receives differs from what a human reads. LLMs don't interpret escape sequences reliably, and control codes wedged into "the position where the word error should be" were blocking recognition.

Fix:

TSC_ARGS="--noEmit --pretty false ..."

Just add --pretty false to the arguments. This makes tsc's output plain text. A good rule to remember: always add --pretty false in hooks and CI environments. Terminal output is for humans to read; program-processed output doesn't need it.

⑤ Concurrent launches corrupted `.tsbuildinfo`

Symptom: After using the hook for a while, tsc suddenly started emitting Cannot read file 'node_modules/.cache/tsc-hook.tsbuildinfo'. Deleting the file and retrying fixes it, but it breaks again a few hours later.

Cause: Claude Code sometimes executes multiple tools in parallel. When the agent issues "write file A" and "write file B" simultaneously, both PostToolUse hooks launch at the same time. When two tsc --incremental processes try to write to the same tsc-hook.tsbuildinfo file concurrently, the file ends up corrupted in a half-written state.

Fix: The current script doesn't include full mutual exclusion (flock etc.). Instead, the policy is the 60-second timeout: "if multiple tsc processes run for a long time, let the OS reap them." The comment says as much: # 並行 hook 実行による多重 tsc を防止 (prevent multiple tsc from concurrent hook execution).

The perfect solution is to use flock.

(
  flock -x 200
  # tsc 実行
) 200>"$TSBUILDINFO.lock"

But adding this means more verification on macOS (BSD flock and GNU flock behave differently). In current operation, if .tsbuildinfo gets corrupted you delete it and it regenerates automatically next time, so I tolerate the loose timeout-based mitigation. It's a judgment call that prioritizes operational simplicity over precision.

Even if tsbuildinfo breaks, Claude Code itself doesn't stop. If the hook returns without type errors, it proceeds to the next step. At worst the impact is limited to "type checking is skipped for that one run." It's a concrete example of the principle that you don't need to complicate code for non-fatal errors.

Stumbling Points (Real Landmines from Setup to Operation)

The middle section covered five: "pipe makes rc always 0," "macOS stat syntax difference," "exit 1 stopped the agent," "ANSI code pollution," and "tsbuildinfo corruption from concurrent launches." Here's a rapid-fire list of everything else that's easy to trip over.

Forgetting chmod +x makes the hook silently ignored

Just placing the script doesn't make it run. Until you grant execute permission with chmod +x ~/.claude/hooks/post_tsc_check.sh, Claude Code quietly skips the hook. No error or warning appears, so you sit in a state of "I configured the hook but no type check comes back." To verify, just run ls -la ~/.claude/hooks/ and check it shows rwxr-xr-x. This is the most common cause of getting stuck on day one.
A #!/bin/sh shebang means bash syntax won't parse in some places

The ${SIZE:-0} parameter expansion and command -v work in /bin/sh too, but the moment you add bash-only syntax while extending the script, it breaks. That's why the first line of the actual file is #!/bin/bash. It's safest to use #!/bin/bash from the start.
If the current directory isn't the project root, the tsconfig.json check misses

PostToolUse hooks launch in the cwd at the time the agent executed the tool. If the agent is editing packages/api/src/foo.ts and the cwd is packages/api/, no problem — but if it's still at the root, it can't find the subpackage's tsconfig.json and exits immediately with exit 0. In a monorepo layout you need to reinforce this either by specifying the expected tsconfig path explicitly with --project, or by doing cd "$(git rev-parse --show-toplevel)" at the top of the hook to move to the root before running.
In a monorepo, tsc picks up an unexpected tsconfig.json

With multiple tsconfig.json files scattered under packages/, which one gets referenced changes with the cwd. Cases arise where you intended to use the root tsconfig but a subpackage's tsconfig gets picked up, shifting the scope of type error detection. Two options: specify an absolute path in TSC_ARGS like --project $(pwd)/tsconfig.json, or build a separate monorepo-aware hook.
Dropping --noEmit generates js files on every hook run

If you delete --noEmit while manually tweaking the incremental arguments, tsc writes out .js files. Build artifacts get overwritten on every hook, polluting git status. Worse, the agent can enter an infinite loop: "file changed → hook fires → js generated → file changed…" Keep --noEmit fixed at the front of the argument template.
Whitespace in the TSBUILDINFO path splits the argument

If quotes are missing from --tsBuildInfoFile $TSBUILDINFO, the moment the project path contains a space, the argument passed to tsc splits and errors out. If you're using the script as-is without quotes, in an environment where the path contains spaces you need to add double quotes: --tsBuildInfoFile "$TSBUILDINFO".
npx startup cost piles up in high-frequency sessions

npx tsc resolves the existence of node_modules/.bin/tsc before every run. The resolution itself is around 0.1–0.3 seconds, but when the agent edits dozens of files in one session, it adds up. In environments where node_modules definitely exists, you have the option of pathing directly to ./node_modules/.bin/tsc. But that reduces resilience for the initial setup when node_modules is absent, so it's not suited to scripts shared with CI.
Neither gtimeout nor timeout exists in the CI environment

Minimal images like Alpine Linux sometimes don't include timeout. TIMEOUT_CMD="" stays empty and tsc runs with no 60-second cutoff. CI type checks are usually managed in a dedicated job, but if you use the same script in CI, install timeout beforehand with something like RUN apk add --no-cache coreutils.
head -30 truncates errors midway and the agent loses sight of the root cause

In projects with cascading type errors, the core is concentrated in the first 30 lines, but in cases like circular reference errors where "the root cause is further down," 30 lines can fall short. In that case, an effective approach is to combine it with an auxiliary hook that leaves the hook output in /tmp/tsc-last.log via tee and has the agent read the full thing with cat /tmp/tsc-last.log. Keep the hook body simple, and supplement through a separate channel when information runs short — a separation-of-concerns mindset.
Applying the 200KB threshold uniformly to every project

200KB is a value derived from measurements on one specific project, closet-os. Change the project size, dependency tree depth, or machine specs, and the inversion point changes too. Get the real numbers for your own project with the following commands.

  ls -lh node_modules/.cache/tsc-hook.tsbuildinfo  # キャッシュの現サイズ
  time npx tsc --noEmit                             # 通常モードの実測
  time npx tsc --noEmit --incremental \
    --tsBuildInfoFile node_modules/.cache/tsc-hook.tsbuildinfo  # インクリメンタルの実測

Take the cache size at the point where second-and-later incremental runs become "slower than normal mode" as your threshold, and you'll have a boundary value specific to your project.

Best Practices

Here are the "I wish I'd done this from the start" principles that solidified through repeated implementation and failure.

1. Write the tsconfig.json check on line one

[ -f "tsconfig.json" ] || exit 0

This is the defensive line that brings hook launch cost to zero in non-TypeScript projects. It's the precondition for everything that follows, so it always goes on the first line.

2. Never remove --noEmit --pretty false — they're a set

--noEmit prevents file generation, and --pretty false eliminates ANSI escape codes. Hook output becomes agent input, so machine-processed text doesn't need color codes. Remove either of these two options from the argument template and both quietly cause problems.

3. Always return exit 0

The hook's job is "to pass information," not "to stop processing." The moment you return a type error with exit 1, the hook turns from an inspection tool into an obstacle. Write the error content to stdout and return exit 0, and the agent receives the information while proceeding autonomously into a fix loop.

4. Use 2>&1 | head -30 as a set

tsc errors go to stderr. Without merging them into stdout via 2>&1, $result gets nothing. head -30 is buffer protection for when errors explode into hundreds of lines. Either one alone is incomplete.

5. Check for timeout in the order gtimeout → timeout → none

TIMEOUT_CMD=""
if command -v gtimeout >/dev/null 2>&1; then
  TIMEOUT_CMD="gtimeout 60"
elif command -v timeout >/dev/null 2>&1; then
  TIMEOUT_CMD="timeout 60"
fi

macOS has no /usr/bin/timeout. Check with command -v in this order before assigning, and the same script runs on Linux, macOS, and CI.

6. Treat rc=124 separately from type errors

The exit code when timeout/gtimeout kills the process is 124. This is the information "tsc didn't stop," which means something different from a type error. By printing a dedicated message and returning exit 0, you can tell the agent about suspected concurrent launches while keeping the flow going.

7. Pin .tsbuildinfo to node_modules/.cache/

TSBUILDINFO="node_modules/.cache/tsc-hook.tsbuildinfo"

Putting it in the project root increases .gitignore maintenance. node_modules/.cache/ is already excluded in most projects, so no extra configuration is needed. Giving the file a distinctive name like tsc-hook to separate its namespace from the app's own build cache is also important.

8. Measure the 200KB guard threshold in your own project before setting it

Check the current cache size with stat -f %z, compare time npx tsc against time npx tsc --incremental, and find the inversion point yourself. Don't reuse the magic number; put your measured boundary value in place of 204800 (200KB).

9. Use -f %z for macOS stat

Linux's -c %s is invalid on macOS. Combined with 2>/dev/null || echo 0, you get the zero fallback for a missing file in one line. If you need both Linux and macOS support, wc -c < also works, but it reads the whole file on large files, so stat is slightly faster.

10. Always verify with a manual run after installing

cd /path/to/your/project
bash ~/.claude/hooks/post_tsc_check.sh

Run it manually before going through Claude Code. Permissions, paths, stat options, tsconfig.json detection — this one command surfaces all of them. It's vastly faster than debugging through Claude Code. Once confirmed clean, enable it as the production hook.

11. Temporarily disable the hook around big dependency changes

Right after adding a new library to package.json, tsc runs a full analysis and the hook blocks for 30–60 seconds. In those moments the right call is to temporarily disable it with chmod -x ~/.claude/hooks/post_tsc_check.sh, verify yourself with npm install && npx tsc --noEmit, and then restore chmod +x. Rather than clinging to full automation, the flexibility to toggle the hook by situation is what keeps operations stable.

12. Keep the script within 61 lines

The actual file is 61 lines including comments. Line count grows every time you add a feature, but if you hold the responsibility boundary of "the hook's job is only to check and report," it won't bloat. If processing gets complex, that's a sign to extract it into a separate dedicated script rather than the hook. "The clarity of 100 lines over 1,000" is achievable at this scale.

Summary

What I built is a 61-line shell script. That alone changed the closet-os hook execution time from "3.6x slower cold" to "1–3 seconds on diffs."

The takeaways fit in three lines.

--incremental backfires on large projects. The size of .tsbuildinfo is the indicator, and on closet-os the inversion point was around 200KB.
A one-line guard that automatically falls back to normal mode when the size from stat -f %z exceeds 204800 (200KB) solves it.
The hook is an "information channel," not a "control channel." Always return exit 0, and pass type errors to the agent via stdout.

With this structure running, every time Claude Code introduces a type error, the "=== TypeScript型エラー検出 ===" feedback comes back and the agent autonomously runs the fix loop. The lag for a human to notice a type error is zero. It's one of the unglamorous parts supporting a ¥1.2M-a-month autonomous setup, but precisely because it's unglamorous, it has kept running stably for months.

If you're feeling that --incremental is heavy on a project around the size of closet-os right now, start by running ls -lh node_modules/.cache/tsc-hook.tsbuildinfo. If it's over 200KB, you can fix it today.

Where does the inversion point land on your project — and what size does your .tsbuildinfo hit before incremental stops paying off?

The full picture of the setup, the breakdown of the ¥1.2M/month, and a 30-day walkthrough are in a paid note (Japanese)
📕 Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

DEV Community: Lily

4 Days, 3 Wasted Calls Per Run: My Retry Loop Mistook a Quota-Limit Message for a 'Too Short' Article

The problem: 4 straight days (9/13–9/16) of 3 failures per run

The fix: grep for the limit message and abort after 1 pass

The pitfall I stepped in: the sibling scripts still don't have it

Summary

One Line of mv Kills the 4x Duplicate Run After Recovery: Making launchd Queue Processing Idempotent

Why This Design Works

The Fork Between "Fixing by Habit" and "Fixing the Environment"

launchd Doesn't Know About "Fighting Over Files"

Backlog Plus Simultaneous Recovery Is What Surfaces the Race

Why a Lock File Isn't Enough

The Guard Philosophy Shown by autolike-plist-reconcile.sh

The Overall Flow

Architecture Diagram: From Race to Resolution

Why mv Is Atomic

Wiring It into the Pickup Script

Connecting with com.lily.codex-note-funnel.plist

Implementation Details

The Full Script: The Order for Stacking Defensive Layers

Why Override PATH at the Top

Why set -uo pipefail and || exit 0 Must Go Together

The "Double Check" Idiom Demonstrated by autolike-plist-reconcile.sh

Design Rationale for Startup Recovery

Where I Got Stuck

Stuck #1: The Race Was Gone, but the Error Log Kept Growing

Stuck #2: Files Kept Piling Up in processing/ While done/ Looked Empty Forever

Stuck #3: Works When Run from the Terminal, Silently Does Nothing from launchd

Stuck #4: ls *.json Returned Exit Code 2 on "No Files" and Crashed the Script

Stuck #5: It Took a Full Day to Notice funnel-pm Was Firing 4x

Pitfalls

Environment and Configuration

Traps Specific to the mv Claim Pattern

Operations and Monitoring Traps

Best Practices

Summary

2 Pitfalls in Priority Probes: Letting One Real Request Through an Open Circuit Breaker

The problem: the circuit stops every job equally, but posting can't wait

The design: give --priority jobs one attempt every 30 minutes

Pitfall 1: "one every 30 minutes" is a counter shared by all priority jobs

Pitfall 2: when the probe fails, it silently lands as RAN exit≠0

Summary

13 Days of Silence: How One Bad Draft Froze an Entire DM Lane

The problem: the exit code was there, but nobody saw it

Root cause: turning "one bad record" into "everything stops"

Fail-closed is for stopping that one record

1. Record the dropped row in the ledger and treat it as exhausted

2. Move anyone with a failure history to the back of the queue

Monitoring: an exit code alone doesn't mean "it's working"

Pitfalls I hit

Takeaways

3 Weeks of Silent Failure: How One Missing `--model` Flag Drained My Interactive Quota Across 4 Jobs

Why This Setup Works

The Overall Flow

The plist Implementation

Four Lanes Surfaced at Once on September 3, 2026

Implementation Details

Where --model Goes and How to Write It

Designing strict_quota_message—The 400-Character Wall

The Postcondition Gate—"There Was Output" Is Not Success

A grep Gate That Mechanically Catches the Omission

Where I Got Stuck

Three Weeks Without Noticing, Until Stock Hit 2

The Wording Changed and the Circuit Breaker Let It Straight Through

The Circuit Breaker Stopped the Posting Jobs Too

Exit 0, but the Output Was a Quota Message

Pitfalls

Best Practices

Summary

5 Pitfalls I Hit Building a Daemon That Auto-Approves Claude Code's Computer Use Dialog

The problem: pressing "Allow for this session" by hand, every time

Implementation: poll every Terminal tab, and don't misfire

The trap I fell into: launching from launchd hangs silently

The workaround: launch as a child process of Terminal

Pitfalls I hit

Summary

4 Automation Chrome Processes Were Killing My Real Browser: Bundle-ID Isolation and a 6-Hour Self-Repair Loop

Why This Setup Works

The Overall Flow

Bundle Creation and Version Management

Why `set -uo pipefail` and `|| exit 0` Must Go Together

Stuck #4: `ls *.json` Returned Exit Code 2 on "No Files" and Crashed the Script

Stuck #5: It Took a Full Day to Notice `funnel-pm` Was Firing 4x

The design: give `--priority` jobs one attempt every 30 minutes

Pitfall 2: when the probe fails, it silently lands as `RAN exit≠0`

Where `--model` Goes and How to Write It

Designing `strict_quota_message`—The 400-Character Wall

`find` Was Taking 4 Minutes

preflight Was `pkill -9`-ing the Human's Chrome Too

Why the `.retired` rename became a habit

Adding just 4 elements to `ProgramArguments`

The `bootout → bootstrap` retry pattern

`launchctl print` is the only way to see the real state

② `bootstrap` returned `5: I/O error` and erased the job

③ I trusted `launchctl list` and declared all 44 jobs "done"

④ A script was unconditionally overwriting the `CLAUDE_BIN` environment variable

`normalize_caption_key` — why it ended up this way