DEV Community

Lily
Lily

Posted on Originally published at dev.to

5 Content Lanes, One Watchdog: How I Stopped Wondering If My Automation Still Runs

Every morning I used to open my logs and ask the same question: did it actually run last night? The scripts always reported success. The articles were sometimes 186 bytes of an error message. This is how I replaced that daily anxiety with a single shell script.

Why this design works

When I first built a content generation pipeline, the first problem I hit was this: I thought it was running, but it had actually stopped.

launchd fires the script every morning, but it starts before the network is up and exits silently. The API times out with no response, yet a log file still exists. Claude's token budget runs dry, and the error message flows straight into the output file, leaving the body at zero bytes. All of these failures leave behind nothing but the fact that "the script ran."

What does it mean to build an environment rather than do work? My answer was a design principle: prove completion by the existence of a file. Not what the script wrote to the log — only whether ~/.claude/logs/.article-daily-done-20260710 exists is treated as truth. That's the essence of the done-marker pattern.

content-watchdog.sh is what happens when you extend that idea across all five content lanes. It gets invoked multiple times a day and does one simple job: check the done-markers for every lane, and restart only the ones that are missing. It doesn't break precisely because it's simple.

Automation gets stuck for operational reasons more often than technical ones. If you design on the assumption that "it worked yesterday, so it'll work today," it quietly dies on the morning the Wifi isn't connected, at midnight when the battery is at 3%, at the end of the month when the budget runs out. Having a watchdog freed me from the nagging worry that "it should still be running today." The reason I can keep 10 iOS apps going in parallel and hold ¥1.2M/month in revenue is that the time spent on verification is as close to zero as it gets.

The overall flow

Here's the relationship between the watchdog and the individual lane scripts.

launchd (複数スロット)
    │
    └─→ content-watchdog.sh (sweep モード)
            │
            ├─ acquire_lock()       # mkdir 競合ロック
            │       └─ ~/.claude/locks/content-watchdog.lockd/
            │
            ├─ for lane in article note maker series ameba
            │       │
            │       ├─ done_lane()  # done-marker / .done ファイルを確認
            │       │
            │       └─ [未完了なら] run_capped 1800 bash <script> apply
            │                │
            │                └─ 各レーンスクリプトが自前のdone-markerを立てる
            │                       例: ~/.claude/logs/.article-daily-done-20260710
            │
            └─ [全レーン完了なら] send_heartbeat_once()
                    └─ Discord通知 + ~/.claude/logs/.content-watchdog-heartbeat-20260710
Enter fullscreen mode Exit fullscreen mode

Let's read through the actual code in order.

The done-marker check logic

The done_lane() function has different check logic for each of the five lanes.

done_lane() {
  local lane="$1"
  local hit
  case "$lane" in
    article)
      [ -f "$LOG_DIR/.article-daily-done-$TODAY" ]
      ;;
    note)
      hit="$(find "$LOG_DIR/note-daily" -maxdepth 1 -name "$TODAY-*.done" -print -quit 2>/dev/null || true)"
      [ -n "$hit" ]
      ;;
    maker)
      hit="$(find "$LOG_DIR/maker-daily" -maxdepth 1 -name "$TODAY-*.done" -print -quit 2>/dev/null || true)"
      [ -n "$hit" ]
      ;;
    series)
      [ -f "$LOG_DIR/.series-daily-done-$TODAY" ]
      ;;
    ameba)
      local ad td f
      ad="$HOME_DIR/Desktop/Article/ameba"
      td="$(date +%F)"
      grep -rlq "created:.*$td" "$ad" 2>/dev/null && return 0
      for f in "$ad"/*.md; do
        [ -f "$f" ] || continue
        [ "$(stat -f %Sm -t %F "$f" 2>/dev/null)" = "$td" ] && return 0
      done
      return 1
      ;;
  esac
}
Enter fullscreen mode Exit fullscreen mode

article and series are managed with a single hidden file (.article-daily-done-20260710). note and maker use a 20260710-*.done pattern, a design that allows multiple files to exist. Only ameba has no done-marker and instead checks the actual artifact directly (the created: metadata in .md files, or the mtime). This is a fallback implementation, needed because the ameba script has no done-marker spec of its own, and it functions as "a realistic compromise for wiring an existing script that lives outside the design into the watchdog."

When does the done-marker get set?

Looking at the implementation of article-daily-stock.sh, the timing of the done-marker is clear.

DONE_MARKER="$HOME/.claude/logs/.article-daily-done-${TODAY}"
Enter fullscreen mode Exit fullscreen mode

The path is fixed at the top of the script, and after the article body has passed generation, validation, stock placement, and secret scanning, the marker is set before the git push (line 453).

# 生成成功=この時点で当日doneを確定する。以降のgit pushはbest-effort(失敗しても生成は成功扱い)
# なので、git失敗でマーカー未設定→翌スロットで重複生成、という事故を防ぐためここで先に立てる。
touch "$DONE_MARKER"
Enter fullscreen mode Exit fullscreen mode

The comment says it's "to prevent the accident of git failure → marker unset → duplicate generation in the next slot." As long as push is best-effort, judging done by push success causes duplicate generation. The design principle that "generation and delivery are independent responsibilities" shows up right here.

Contention locking with mkdir

Since the watchdog itself can be launched from multiple slots, it uses mkdir for mutual exclusion.

acquire_lock() {
  if mkdir "$LOCKDIR" 2>/dev/null; then
    printf '%s\n' "$$" > "$LOCKDIR/pid"
    trap release_lock EXIT INT TERM
    return 0
  fi

  local now mod age
  now="$(date +%s)"
  mod="$(stat -f %m "$LOCKDIR" 2>/dev/null || printf '%s\n' "$now")"
  age=$((now - mod))
  if [ "$age" -ge 1800 ]; then
    log "lock stale age=${age}s; taking over"
    rm -rf "$LOCKDIR"
    if mkdir "$LOCKDIR" 2>/dev/null; then
      printf '%s\n' "$$" > "$LOCKDIR/pid"
      trap release_lock EXIT INT TERM
      return 0
    fi
  fi

  log "lock held; skip"
  exit 0
}
Enter fullscreen mode Exit fullscreen mode

mkdir is an atomic operation at the POSIX level. Even if two processes call mkdir simultaneously, only one succeeds. No flock needed, no Linux/macOS compatibility issues, and the simplicity of exit 0 immediately on failure is its strength.

If the lock has been sitting untouched for 1800 seconds (30 minutes) or more, it judges that "the previous process died abnormally and left the lock behind" and forcibly takes over. The same 1800 seconds is used as the timeout for each lane's script invocation, so the next watchdog won't start unless a healthy watchdog has just finished its run.

article-daily-stock.sh also has its own lock using the same approach.

LOCKDIR="$HOME/.claude/locks/article-daily.lock"
if ! /bin/mkdir "$LOCKDIR" 2>/dev/null; then
  oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || true)
  if [ -n "${oldpid:-}" ] && kill -0 "$oldpid" 2>/dev/null; then
    log "別インスタンス実行中(pid=$oldpid) — skip"; exit 0
  fi
  rm -rf "$LOCKDIR"; /bin/mkdir "$LOCKDIR" 2>/dev/null || exit 0
fi
echo $$ > "$LOCKDIR/pid"
trap 'rm -rf "$LOCKDIR"' EXIT INT TERM
Enter fullscreen mode Exit fullscreen mode

This one adds a PID liveness check. If the previous process is alive it genuinely skips; if it's dead, it force-releases the lock and runs itself. Because the watchdog's lock and each lane script's lock exist as two separate layers, "the watchdog is trying to restart" and "the previous article generation process is still running" don't interfere with each other.

The design of Discord notifications and the heartbeat

The design of notifying on failure but only once a day on success is baked into send_heartbeat_once().

send_heartbeat_once() {
  [ "$MODE" = "sweep" ] || return 0
  if [ -f "$HEARTBEAT_MARKER" ]; then
    log "heartbeat skip already_sent=1"
    return 0
  fi
  notify alerts "💓 content-watchdog heartbeat: 全レーン当日生成済み"
  touch "$HEARTBEAT_MARKER"
  log "heartbeat sent"
}
Enter fullscreen mode Exit fullscreen mode

The watchdog gets invoked many times a day. If all lanes are healthy, it sends a 💓 to Discord only the first time and short-circuits on the marker afterward. Failure notifications, by contrast, fire every time.

if [ -n "$FAILED_LANES" ]; then
  log "result=unhealthy lanes=$FAILED_LANES"
  if [ "$MODE" = "sweep" ]; then
    notify alerts "🚨 content停止: $FAILED_LANES 当日未生成(自己修復不能)"
  fi
Enter fullscreen mode Exit fullscreen mode

If the done-marker still isn't set after attempting auto-repair (restarting inside run_lane, then checking done_lane() once more), it tells Discord that manual intervention is required.

Looking at the internals of run_lane, the "restart → recheck" cycle fits into a single function.

run_lane() {
  local lane="$1"
  local script rc
  script="$(script_for "$lane")"

  if done_lane "$lane"; then
    log "lane=$lane status=healthy action=skip"
    return 0
  fi

  if [ ! -f "$script" ]; then
    log "lane=$lane status=missing script=$script"
    return 1
  fi

  log "lane=$lane status=missing-done action=reinvoke script=$script timeout=1800s"
  if [ "$lane" = "ameba" ]; then
    run_capped 1800 bash "$script" >> "$LOG" 2>&1
  else
    run_capped 1800 bash "$script" apply >> "$LOG" 2>&1
  fi
  rc=$?
  log "lane=$lane reinvoke_exit=$rc"

  if done_lane "$lane"; then
    log "lane=$lane status=healthy-after-reinvoke"
    return 0
  fi

  log "lane=$lane status=failed-after-reinvoke"
  return 1
}
Enter fullscreen mode Exit fullscreen mode

The flow is: check with done_lane → skip if fine → restart if not → check with done_lane again. The exit code of the restart (rc) is written to the log, but it isn't used for the decision. The strictness of "even with exit code 0, no done-marker means failure" is what catches the case where a script spits an error into the output file and exits normally.

Implementation details

Verifying the artifact is "an article with actual content" in three layers

I explained earlier that run_lane() uses the done-marker as its trust basis. So what gets checked before the done-marker is set? The validation layer in article-daily-stock.sh is the answer.

First, article_ok() checks the file's existence and its contents.

MIN_ARTICLE_BYTES=1200

article_ok() {
  local f="$1"
  [ -s "$f" ] || return 1
  [ "$(stat -f%z "$f" 2>/dev/null || echo 0)" -ge "$MIN_ARTICLE_BYTES" ] || return 1
  grep -qE '^title:' "$f" || return 1
  grep -qiE 'request timed out|不明な商品|TODO: *本文|\(生成失敗\)' "$f" && return 1
  return 0
}
Enter fullscreen mode Exit fullscreen mode

Four guards are chained in series: does the file exist, is it at least 1200 bytes, does the frontmatter have a title line, and is it free of timeout wording or generation-failure phrases? Keep that last grep -qiE in mind — it connects directly to a story about getting stuck later.

Thumbnail verification is handled by a separate function, thumb_ok().

thumb_ok() {
  local f="$1" w; [ -s "$f" ] || return 1
  w=$(sips -g pixelWidth "$f" 2>/dev/null | awk '/pixelWidth/{print $2}')
  [ -n "$w" ] && [ "$w" -ge 2000 ] 2>/dev/null
}
Enter fullscreen mode Exit fullscreen mode

sips is macOS's built-in image tool. Anything under 2000 pixels wide isn't accepted as a thumbnail. Even if the file exists, zero-byte or corrupted PNGs get rejected.

audit_repair() uses these two functions to re-verify the entire stock on every launch (it's a 312-line monster of a function, so I'll quote only the key parts).

audit_repair() {
  for f in "$STOCK_ART"/[0-9][0-9]-*.md; do
    if [ "$t_ok" = "missing" ] && [ "$a_ok" = "ok" ]; then
      if regen_thumb "$slug" "$no" "$f"; then
        t_ok=ok; fixed=$((fixed+1)); log "FIX: サムネ再生成 $no-$slug"
      fi
    fi
    [ "$a_ok" = "broken" ] && broken+=("$no-$slug")
  done
  for nb in "${broken[@]:-}"; do
    n=$(bump_attempt "$slug")
    if [ "$n" -le "$MAX_ATTEMPTS" ]; then
      jq --argjson t "$topic" '[$t] + (map(select(.slug != ($t.slug))))' "$QUEUE" > ...
      log "REQUEUE: $slug 再生成キューへ投入(試行 $n/$MAX_ATTEMPTS)"
    else
      needhuman+=("$nb (再生成${n}回失敗)")
    fi
  done
}
Enter fullscreen mode Exit fullscreen mode

MAX_ATTEMPTS=2 is declared as a constant (line 136). If a thumbnail is broken it's regenerated automatically; if the body is broken it's returned to the queue for regeneration up to two times, and on the third it notifies a human and stops. This is the self-repair that keeps things from being "left stuck."

audit_repair() runs every time, before article generation (line 246). Even when the generated flag causes today's generation to be skipped, the audit keeps running. The core of the design is that new generation and stock quality assurance run on independent cycles.

audit_repair
if [ "$MODE" = "audit" ] || [ "$SKIP_GEN" = "1" ]; then
  log "===== article-daily done(audit$([ "$SKIP_GEN" = "1" ] && echo '+gen-skipped')) ====="
  exit 0
fi
Enter fullscreen mode Exit fullscreen mode

Killing three startup-timing traps with three mechanisms

Even when launchd calls the script every morning, the environment right after boot is less stable than you'd think. There are three mechanisms.

1. caffeinate: keep the Mac from sleeping

if [ -z "${CAFFEINATED:-}" ]; then
  exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@"
fi
Enter fullscreen mode Exit fullscreen mode

It uses exec to relaunch itself wrapped in caffeinate. Passing the environment variable CAFFEINATED=1 prevents a double exec. -i is the flag that prevents idle sleep while the process is alive, and -s prevents system sleep. Without this, the Mac falls asleep mid-generation when running on battery late at night.

2. Network wait: don't call the API before Wifi is up

if [ "$MODE" != "audit" ]; then
  for _ in $(seq 1 18); do
    /usr/bin/nc -z -G 3 1.1.1.1 443 2>/dev/null && break; sleep 5
  done
fi
Enter fullscreen mode Exit fullscreen mode

It uses nc to check connectivity to port 443 on 1.1.1.1, repeating up to 18 times (90 seconds) until it connects. -G 3 is a 3-second connect timeout. Since launchd sometimes calls the script right after the Mac boots, this handles the case where hitting Claude's API before the Wifi connection is established wipes everything out. Audit-only mode skips it because it doesn't use Claude.

3. Budget check: do nothing if tokens are exhausted

BUDGET=$(~/.claude/scripts/token-budget-advisor.sh --short 2>/dev/null || echo "n/a")
log "budget: $BUDGET"
if [ "$MODE" != "audit" ] && echo "$BUDGET" | grep -qE '🔴|critical|cap-near'; then
  log "ABORT: budget critical — 次スロットで再試行"; exit 0
fi
Enter fullscreen mode Exit fullscreen mode

token-budget-advisor.sh returns the token consumption status, and if it contains 🔴 or critical, the script exits without doing anything. Even if the budget runs out at the end of the month, the watchdog automatically restarts things at the next slot (after the monthly reset), so there's nothing to do by hand.

The run_capped pattern that absorbs gtimeout's presence or absence

run_capped() in content-watchdog.sh looks simple at a glance, but it's an important wrapper that absorbs environment differences.

timeout_bin() {
  if [ -x /opt/homebrew/bin/gtimeout ]; then
    printf '%s\n' /opt/homebrew/bin/gtimeout
  else
    command -v gtimeout 2>/dev/null || true
  fi
}

run_capped() {
  local limit="$1"
  shift
  local tb
  tb="$(timeout_bin)"
  if [ -n "$tb" ]; then
    "$tb" "$limit" "$@"
  else
    log "gtimeout unavailable; running without timeout: $*"
    "$@"
  fi
}
Enter fullscreen mode Exit fullscreen mode

It checks the homebrew absolute path first, and if that's not there, searches PATH with command -v. If neither works, it runs without a timeout while logging that fact. The key point is || true, which keeps it from erroring — preventing the accident of "the entire watchdog dies because gtimeout is missing." article-daily-stock.sh has a run_to helper built on the same philosophy.

TIMEOUT_BIN="/opt/homebrew/bin/gtimeout"
[ -x "$TIMEOUT_BIN" ] || TIMEOUT_BIN="/opt/homebrew/bin/timeout"
[ -x "$TIMEOUT_BIN" ] || TIMEOUT_BIN=""
run_to() { local s=$1; shift; if [ -n "$TIMEOUT_BIN" ]; then "$TIMEOUT_BIN" --kill-after=30 "$s" "$@"; else "$@"; fi; }
Enter fullscreen mode Exit fullscreen mode

This one also adds --kill-after=30, giving a 30-second grace period before sending SIGKILL after the timeout. Claude's process doesn't die instantly when it receives SIGTERM, so this two-stage approach is necessary.

Auto-refilling topics so it doesn't stop when the queue empties

Article topics are stored in ~/.topic-queue.json. When this queue empties, the script doesn't stop — it uses claude -p to automatically draft one topic and add it to the queue.

QLEN=$(jq 'length' "$QUEUE" 2>/dev/null || echo 0)
if [ "$QLEN" -eq 0 ]; then
  log "queue 空 → 実作業からネタ自動立案"
  # ...長いプロンプト組み立て...
  TOPIC_JSON=$(run_to 600 "$CLAUDE" -p "$REPLENISH_PROMPT" \
    --model "${ARTICLE_MODEL:-claude-sonnet-4-6}" --effort high \
    --output-format text --allowedTools "Read,Grep,Glob,Bash" --max-turns 20 2>>"$LOG")
Enter fullscreen mode Exit fullscreen mode

--max-turns 20 matters. It's the cap on how many round-trip turns Claude gets to Read actual files before producing output for topic planning. Without it, Claude investigates too thoroughly and burns through tokens.

It also checks whether the drafted topic duplicates an existing slug.

if used_slugs | grep -qx "$NEW_SLUG"; then
  log "ABORT: 立案slug '$NEW_SLUG' が既出 → 次スロット再試行"; exit 0
fi
Enter fullscreen mode Exit fullscreen mode

used_slugs() collects slugs from four sources — the queue, the completed list, existing article files, and coverage.json — and checks for duplicates. Without it, you end up mass-producing articles on exactly the same topic (this actually happened; more on that below).


Where I got stuck

"The script ran" — and the article was 186 bytes

This was the first failure. The launchd log recorded a normal exit. The file existed. But when I opened it, this was all that was inside.

---
title: "Claude Codeを使った自動化"
emoji: "🤖"
type: "tech"
published: true
---

request timed out
Enter fullscreen mode Exit fullscreen mode

The autonomous Claude Code session was cut off by a 30-second network timeout, and that error message flowed into the output file. The script's exit code was 0. The [ -s "$ART" ] check passed too. Because this state was treated as "success," the next morning's watchdog decided "done-marker present, skip" and the article was never repaired.

The fix was adding article_ok(). I added grep -qiE 'request timed out' to the validation conditions; if the phrase is present in the body, it's treated as a generation failure and the file is deleted. The script exits without setting the done-marker, so the watchdog restarts it at the next slot.

Lesson: exit code 0 is proof that the script ran, not proof that the output is correct. You can't tell whether generation succeeded without looking at the contents.

The era when I set the done-marker after git push

In the original implementation, a successful git push was the condition for setting the done-marker. It sounds logical — the idea that "done includes publishing."

But it broke at the end of the month. I hit GitHub's API rate limit and pushes failed repeatedly. The watchdog then decided "no done-marker, restart" at every slot, and three articles on the same topic were generated on the same day. The contents were subtly different. The filenames had sequential numbers. It was a mess.

A comment in the code records that history (line 453).

# 生成成功=この時点で当日doneを確定する。以降のgit pushはbest-effort(失敗しても生成は成功扱い)
# なので、git失敗でマーカー未設定→翌スロットで重複生成、という事故を防ぐためここで先に立てる。
touch "$DONE_MARKER"
Enter fullscreen mode Exit fullscreen mode

Now the queue pop, the record into the done-queue, and the marker placement all happen before the git push. Push is nothing more than "best-effort delivery to the Zenn repo," an independent responsibility from article generation and stock placement. Just in case, touch "$DONE_MARKER" is also called again right after the git push (line 468). It's an idempotent operation, so the cost is zero.

The Mac slept at midnight and article generation died halfway

One night while running my MacBook Air on battery, article generation kicked off at 2 AM and the OS went to sleep. launchd started the process, but disk I/O stopped, the Claude session was interrupted, and the article ended mid-way. The file exists, it's 500 bytes, but the body is cut off. article_ok() rejected it for being "under 1200 bytes" so it was regenerated the next day — but I didn't notice until then.

After adding caffeinate, the Mac no longer sleeps while article generation is running. Because of the -s flag (system sleep prevention), it doesn't sleep even with the lid closed. "Automatic generation every morning" simply didn't hold together without this mechanism.

Only the ameba lane had no done-marker

The generate.sh for ameba wasn't written by me; it was an existing project I bolted the watchdog onto afterward. That script had no done-marker mechanism.

Trying to check for .ameba-daily-done-20260710 the same way as the other four lanes, that file would never exist. The watchdog judged "not done" every time and kept restarting.

So I had no choice but to give the ameba branch of done_lane() a different check method.

ameba)
  local ad td f
  ad="$HOME_DIR/Desktop/Article/ameba"
  td="$(date +%F)"
  grep -rlq "created:.*$td" "$ad" 2>/dev/null && return 0
  for f in "$ad"/*.md; do
    [ -f "$f" ] || continue
    [ "$(stat -f %Sm -t %F "$f" 2>/dev/null)" = "$td" ] && return 0
  done
  return 1
  ;;
Enter fullscreen mode Exit fullscreen mode

It greps for whether an md file with today's created: metadata exists, and if not, uses stat to check whether there's a file with today's mtime. A two-stage fallback. I wrote in a code comment that it's "a realistic compromise for pulling an existing script that lives outside the design into the watchdog," and that's genuinely what it is — the ideal would be to add a done-marker on the generate.sh side. But I compared the cost of touching an existing script against the cost of running with a fallback, and chose the fallback.

The day 10 articles lined up on the same topic

This is from before I added slug duplicate checking. The queue emptied and the automatic topic refill ran. The slug Claude produced was claude-code-automation. It already existed in the done-queue. With no duplicate check, it was added to the queue, and the next day the same topic was generated again. When I checked 10 days later, 01-claude-code-automation.md through 10-claude-code-automation.md were all lined up. The contents differ slightly each time.

The used_slugs() function was added after this accident. By consolidating four sources (queue, completed, existing files, coverage.json), there's no gap no matter when the duplicate check runs.

A leftover lock stopped all of that day's article generation

When a process is force-killed with kill -9, the trap doesn't fire and the lock directory is left behind. When that happens, the next slot's watchdog decides "lock held; skip" and exits immediately.

I noticed two days later, because the 💓 never arrived on Discord. When I checked the log, "lock held; skip" was lined up more than 100 times.

The 1800-second stale detection in acquire_lock() (the code quoted earlier) was added in response to this accident. If the lock has been sitting for 1800 seconds (30 minutes), it forcibly takes over. The longest timeout for article generation is also 1800 seconds, so if a healthy process is running, it will always release the lock within that time. Takeover only happens when the lock is judged stale.

Since putting this fix in, "days when the Discord 💓 doesn't arrive" have been zero. Even when the watchdog gets stuck, a 🚨 flies to Discord, so I can grasp the health of all five lanes just by checking notifications over my morning coffee. One reason I can maintain ¥1.2M/month in revenue while juggling development of 10 iOS apps is this "no need to check" design.

Pitfalls

I covered six "actually got stuck" stories earlier. Here I'll list, in bullet form, the finer traps I noticed while reviewing the implementation. These are all the "looks like it's working, then you realize it's broken" variety.

launchd does not inherit PATH

Line 4 at the top of content-watchdog.sh says this.

export PATH="~/.nvm/versions/node/v24.13.0/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin"
Enter fullscreen mode Exit fullscreen mode

Without this, gtimeout, claude, and jq all go missing. The PATH for a process launched by launchd is only /usr/bin:/bin:/usr/sbin:/sbin. Nine out of ten cases of "it works in the terminal but not under launchd" are this. article-daily-stock.sh solves it with a different approach.

NODE_BIN=$(ls -d "$HOME"/.nvm/versions/node/*/bin 2>/dev/null | sort -V | tail -1)
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$PATH"
[ -n "$NODE_BIN" ] && export PATH="${NODE_BIN}:$PATH"
Enter fullscreen mode Exit fullscreen mode

sort -V sorts by version to dynamically discover the newest nvm binary. The claude binary is resolved right after with a three-stage fallback (command -v~/.local/bin → searching under nvm with ls -t). If none of them find it, it logs ABORT: claude binary not found and does exit 0. It's exit 0 rather than exit 1 because having launchd record it as a "failure" and enter a retry loop would be a problem.

The done-markers for the note/maker lanes use glob search

article and series use hidden files with fixed names (.article-daily-done-20260710), but note and maker are different.

note)
  hit="$(find "$LOG_DIR/note-daily" -maxdepth 1 -name "$TODAY-*.done" -print -quit 2>/dev/null || true)"
  [ -n "$hit" ]
  ;;
Enter fullscreen mode Exit fullscreen mode

It's a 20260710-*.done wildcard pattern. The reason is that note and maker can generate multiple topics in a day, so the topic slug goes into the filename. With a fixed-name done-marker, you'd lose track of which topic completed. -print -quit exits as soon as one match is found, to keep find from scanning everything as the file count grows.

If you write this with the same pattern for all lanes during implementation, the note lane's done-marker will be "never found" forever. You need to understand from the start that the check logic differs per lane.

Handling processes that don't stop on gtimeout's SIGTERM

Claude's process doesn't terminate immediately when it receives SIGTERM. If SIGTERM arrives mid-conversation while it's processing tokens, it takes anywhere from a few seconds to a few dozen seconds to finish processing before exiting. That's why the run_to helper in article-daily-stock.sh attaches --kill-after=30.

run_to() { local s=$1; shift; if [ -n "$TIMEOUT_BIN" ]; then "$TIMEOUT_BIN" --kill-after=30 "$s" "$@"; else "$@"; fi; }
Enter fullscreen mode Exit fullscreen mode

gtimeout's --kill-after=30 is a two-stage approach: "send SIGTERM after the timeout, and if it's still alive 30 seconds later, send SIGKILL." Without it, the post-timeout process lingers like a zombie and trips the double-execution lock when the watchdog starts at the next slot.

If jq isn't found, everything downstream fails silently

Queue processing, coverage.json updates, and the duplicate check in used_slugs() all use jq. article-daily-stock.sh has an explicit check.

command -v jq >/dev/null 2>&1 || { log "ABORT: jq not found"; exit 0; }
Enter fullscreen mode Exit fullscreen mode

Before I added this, running the script in an environment without jq buried jq: command not found deep in the log, the queue appeared to be read correctly but was actually treated as empty, and the automatic topic refill ran every single time.

Logs keep growing and eat disk

The watchdog is launched multiple times a day. Since all of Claude's output flows into the log on every article generation, the log file becomes enormous within days if you do nothing. article-daily-stock.sh rotates once it exceeds 5MB (line 78).

[ -f "$LOG" ] && [ "$(stat -f%z "$LOG" 2>/dev/null || echo 0)" -gt 5242880 ] && mv "$LOG" "$LOG.old"
Enter fullscreen mode Exit fullscreen mode

Rotation isn't implemented on the content-watchdog.log side, so long-term operation requires manual checking. This is one of the unresolved items in my current setup.

The separation of "the audit runs every time, generation runs once a day"

The SKIP_GEN flag is set right after article-daily-stock.sh starts.

SKIP_GEN=0
[ "$MODE" = "apply" ] && [ -f "$DONE_MARKER" ] && SKIP_GEN=1
Enter fullscreen mode Exit fullscreen mode

After that, audit_repair() always runs. It doesn't stop even with SKIP_GEN=1 (line 246).

audit_repair
if [ "$MODE" = "audit" ] || [ "$SKIP_GEN" = "1" ]; then
  log "===== article-daily done(audit$([ "$SKIP_GEN" = "1" ] && echo '+gen-skipped')) ====="
  exit 0
fi
Enter fullscreen mode Exit fullscreen mode

In other words, even when the watchdog calls the script multiple times a day, every call after the first only runs "audit and self-repair of existing stock" and exits. Duplicate generation and stock quality degradation are controlled independently. Without understanding this separation, you won't be able to figure out "why does the watchdog call the script every time but generation only happens once?"


Best practices

From the experience of actually breaking things and fixing them, here are more than 10 design decisions I wish I'd made from the start.

1. Make the proof of completion the existence of a file

Log contents, process exit codes, and printf output only prove "the fact that the script ran." If you design so that only whether the file ~/.claude/logs/.article-daily-done-20260710 exists is treated as truth, then any failure pattern gets restarted at the next slot.

2. Make generation and delivery independent responsibilities

If you judge the done-marker by git push success or failure, network outages and API rate limits look like generation failures. As the comment on line 453 of article-daily-stock.sh shows, set the done-marker at the point generation succeeds and make push best-effort. touch "$DONE_MARKER" is called again after push (line 468), but it's an idempotent operation, so the cost is zero.

3. Validate the contents before setting the done-marker

The done-marker isn't set unless the four guards in article_ok() pass.

article_ok() {
  local f="$1"
  [ -s "$f" ] || return 1
  [ "$(stat -f%z "$f" 2>/dev/null || echo 0)" -ge "$MIN_ARTICLE_BYTES" ] || return 1
  grep -qE '^title:' "$f" || return 1
  grep -qiE 'request timed out|不明な商品|TODO: *本文|\(生成失敗\)' "$f" && return 1
  return 0
}
Enter fullscreen mode Exit fullscreen mode

Only after clearing three things — 1200 bytes, a title line in the frontmatter, and the absence of timeout wording — does it judge that "this holds together as an article." It's the implementation of the principle that exit code 0 proves nothing.

4. Put a cap on auto-repair attempts

MAX_ATTEMPTS=2
Enter fullscreen mode Exit fullscreen mode

This constant is on line 136 of article-daily-stock.sh. It automatically retries regeneration of a broken article up to twice, and on the third it stacks it into the needhuman list and notifies. Repeating regeneration infinitely wastes tokens when the queue is corrupted or Claude can't handle a certain kind of input.

5. Check duplicates against four sources

used_slugs() consolidates four sources.

used_slugs() {
  { jq -r '.[].slug' "$QUEUE" 2>/dev/null
    jq -r '.[].slug' "$DONEQ" 2>/dev/null
    ls "$ARTICLES" 2>/dev/null | sed 's/\.md$//'
    jq -r '.[].slug' "$COVERAGE" 2>/dev/null
  } | sort -u
}
Enter fullscreen mode Exit fullscreen mode

If any one of queue, completed, existing files, or coverage.json is missing, duplicate topics get generated. The "10 articles lined up on the same topic" accident happened because only the coverage check was missing.

6. Take contention locks with mkdir

mkdir is atomic at the POSIX level. flock behaves subtly differently on Linux and macOS, but mkdir's atomicity is guaranteed on both. content-watchdog.sh's lock directory name ends in .lockd (trailing d) to make it explicit that it's a directory.

7. Auto-take-over stale locks at 1800 seconds

Since each lane's timeout is 1800 seconds, a healthy process will always release the lock within that time.

if [ "$age" -ge 1800 ]; then
  log "lock stale age=${age}s; taking over"
  rm -rf "$LOCKDIR"
  ...
Enter fullscreen mode Exit fullscreen mode

This value prevents mistaken takeovers by matching "how many seconds until takeover" with "the lane script's timeout." The point is to keep the numbers consistent.

8. Decide whether to restart based on the done-marker, not the exit code

run_lane() logs the exit code after restarting (rc), but doesn't use it for the decision.

rc=$?
log "lane=$lane reinvoke_exit=$rc"

if done_lane "$lane"; then
  log "lane=$lane status=healthy-after-reinvoke"
  return 0
fi
Enter fullscreen mode Exit fullscreen mode

Exit code 0 but no done-marker counts as needing a restart. Exit code 1 but a done-marker present counts as success. This accurately catches the case where a script flushes an error into the output file and exits normally (timeout wording mixed into the body).

9. Prevent midnight sleep with caffeinate exec

If article generation runs late at night on a battery-powered Mac, the OS goes to sleep.

if [ -z "${CAFFEINATED:-}" ]; then
  exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@"
fi
Enter fullscreen mode Exit fullscreen mode

It uses exec to relaunch itself wrapped in caffeinate. The -s flag prevents system sleep, so it doesn't sleep even with the lid closed. Passing CAFFEINATED=1 as an environment variable prevents a double exec.

10. Always implement a network wait

launchd sometimes calls processes right after the Mac boots. If you hit Claude's API before the Wifi connection is established, all lanes fail.

for _ in $(seq 1 18); do
  /usr/bin/nc -z -G 3 1.1.1.1 443 2>/dev/null && break; sleep 5
done
Enter fullscreen mode Exit fullscreen mode

It waits a maximum of 18 times × 5 seconds = 90 seconds. Capping the connect timeout at 3 seconds with -G 3 means it exits within a few seconds if Wifi is connected. audit mode skips this loop because it doesn't use Claude (line 87).

11. Do the budget check before hitting the API

BUDGET=$(~/.claude/scripts/token-budget-advisor.sh --short 2>/dev/null || echo "n/a")
if echo "$BUDGET" | grep -qE '🔴|critical|cap-near'; then
  log "ABORT: budget critical — 次スロットで再試行"; exit 0
fi
Enter fullscreen mode Exit fullscreen mode

Even if the budget runs out at the end of the month, the watchdog automatically restarts at a slot after the next month's reset. This is a mechanism I added after experiencing "everything stopped at the end of the month and I restarted it by hand."

12. Heartbeat once a day, failure notifications every time

Notifying Discord of every success causes notification fatigue. send_heartbeat_once() narrows it to the first time only via HEARTBEAT_MARKER. Failure notifications (🚨) fire every time.

notify alerts "💓 content-watchdog heartbeat: 全レーン当日生成済み"
touch "$HEARTBEAT_MARKER"
Enter fullscreen mode Exit fullscreen mode

By contrast, failure notification evaluates its condition every time with if [ -n "$FAILED_LANES" ]. It's a design of "quiet success, loud failure." Open Discord in the morning: a 💓 means all lanes are healthy, a 🚨 means manual intervention is needed.

13. For existing scripts like ameba, check the actual artifact rather than a done-marker

When bolting a watchdog onto an existing script after the fact, the cost of modifying that script to add a done-marker is sometimes high. The ameba lane's fallback implementation (a two-stage check of created: metadata and mtime) is a realistic compromise. The ideal is to add the done-marker on the script side, but which to prioritize — "making it work" or "making it perfect" — depends on the situation.


Summary

In this article I walked through the actual code of content-watchdog.sh and article-daily-stock.sh to examine the core of a design that "repairs itself when it gets stuck."

Three ideas sit at the center.

Prove completion by the existence of a file. Not a log string, not an exit code — only whether ~/.claude/logs/.article-daily-done-20260710 exists is treated as truth. This catches every instance of the "the script ran but there's no content" failure pattern.

Make generation and delivery independent responsibilities. If git push success is a condition for the done-marker, a network outage looks like a generation failure. Treat the moment the article is written to ~/Desktop/Article as success, and make delivery best-effort. End-of-month API rate limits no longer affect that day's article generation.

Run the audit every time, and generation only once a day. audit_repair() executes even with SKIP_GEN=1. Even on days when new generation is skipped, quality checks on existing stock and regeneration of broken articles keep running quietly. Without this separation, you're left with a hole: "on a day with the generated flag set, nobody notices when an existing article is broken."

The reason I can maintain ¥1.2M/month in revenue while juggling 10 iOS apps six months after being laid off is this "no need to check" design. Just confirming that a 💓 arrives on Discord every morning tells me all five content lanes are healthy. Before the watchdog existed, I checked logs manually while carrying the worry that "it should still be running today." That time and anxiety cost dropping to zero is what creates the room to focus on other development.


I've put the full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day walkthrough into a paid note.

📕 Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート


Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

Top comments (0)