A circuit breaker that stops everything is easy to reason about — until the one job that can't afford to wait gets stopped along with everything else. In my setup, that job was posting: it fell to a 43% execution rate (231 runs vs. 308 skips) because engagement jobs had burned through the quota first. This post is about the fix inside claude-quota-guard.py — claim_priority_probe and run_job, which let a single real request through while the circuit is still open — and the two pitfalls I hit along the way, one of which silently dropped a daily job for four days (2026-09-13 to 16).
Last time, I wrote about how the gate blocks one row, not the whole batch. Before I get to quota-catchup.py — the script that re-runs everything once it detects quota recovery — I want to look at how the SKIPPED markers it reads are actually produced.
The problem: the circuit stops every job equally, but posting can't wait
When claude-quota-guard.py detects the quota limit, it returns EXIT_CIRCUIT_OPEN (75) until open_until and stops all 15 guarded jobs uniformly. A comment in the code explains why that wasn't enough:
# 🔴 2026-08-21: circuit が開くと全ジョブが一律で止まるため、消費の大半を占める
# 返信/エンゲージ系がクォータを使い切った巻き添えで「投稿」まで停止していた。
# 実測(launchd.log 累計): xpilot.autopost は 231実行/308スキップ＝実行率43%で、
# threadspilot.engage(64%) より優先度が低い扱いになっていた。投稿はその時間帯を逃すと
# 二度と埋まらないので、--priority を付けたジョブだけは circuit が開いていても
# この間隔で1回だけ試行を許す。試行が通ればクォータ回復の早期検知にもなる
# (従来は open_until まで盲目的に待つだけだった)。
(claude-quota-guard.py:18-24)
The measured result: stopping everything uniformly meant the posting job (xpilot.autopost) got caught in the crossfire of the quota-hungry engagement jobs and dropped to a 43% execution rate — effectively lower priority than threadspilot.engage (64%). On top of that, open_until is determined either by "the reset time parsed from the limit message" or by "a 6-hour cooldown" (record_claude_result), so even if the actual quota comes back earlier, the circuit dutifully stays open until that time. For a job like posting, where "if you miss the time slot, it never gets filled," that's not something you can ignore.
The design: give --priority jobs one attempt every 30 minutes
The fix is: "even while the circuit is open, let priority jobs — and only priority jobs — send one real request at a fixed interval." The caller passes --priority when handing a command to run_job.
def run_job(label: str, command: list[str], priority: bool = False) -> int:
if not command:
print("claude quota guard: --job requires a command after --", file=sys.stderr)
return 2
status = circuit_status()
probe_claimed = False
if status["is_open"]:
if not (priority and claim_priority_probe()):
print(
"CLAUDE_QUOTA_JOB_SKIPPED "
f"job={label} reason={status['reason']} remaining={status['remaining_seconds']}s ts={now()}",
file=sys.stderr,
)
return 0
probe_claimed = True
print(
"CLAUDE_QUOTA_JOB_PRIORITY_PROBE "
f"job={label} reason={status['reason']} remaining={status['remaining_seconds']}s ts={now()}",
file=sys.stderr,
)
(claude-quota-guard.py:471-490)
Even with priority set, whether the job can actually attempt anything is decided by claim_priority_probe().
def claim_priority_probe() -> bool:
"""circuit が開いている間、優先ジョブに試行権を1つ渡す。
間隔は全優先ジョブで共有する(=1本が使ったら次の枠まで他も待つ)。上限に本当に
達している間に何本も叩いてもクォータは戻らないため、叩く回数自体を絞る。
"""
if PRIORITY_PROBE_INTERVAL <= 0:
return False
with locked_state() as state:
last = int(state.get("last_priority_probe") or 0)
current = now()
if current - last < PRIORITY_PROBE_INTERVAL:
return False
state["last_priority_probe"] = current
return True
(claude-quota-guard.py:454-468; PRIORITY_PROBE_INTERVAL defaults to 1800 seconds at claude-quota-guard.py:25)
last_priority_probe is a single timestamp stored in locked_state() (a JSON-persisted state guarded by fcntl.flock). The key point is that this value lives per circuit, not per job: it doesn't care who probed, only "has it been 30 minutes since anyone last probed?"
The side that wins the attempt passes CLAUDE_QUOTA_PRIORITY_PROBE=1 in the environment of the child process it launches via subprocess.run.
guard = str(Path(__file__).resolve())
env = os.environ.copy()
env["CLAUDE_AUTOMATION_GUARD"] = "1"
env["CLAUDE"] = guard
env["CLAUDE_BIN"] = guard
if probe_claimed:
# 内側の run_claude に「プローブとして走っている」ことを伝える(これが無いと circuit で即 75 になる)
env["CLAUDE_QUOTA_PRIORITY_PROBE"] = "1"
(claude-quota-guard.py:491-498)
The receiver of this flag is run_claude. When the job command internally invokes the real claude binary, PATH has been rewired so the call goes through the guard itself — which means a second circuit_status() check runs inside the child process.
def run_claude(arguments: list[str]) -> int:
status = circuit_status()
if status["is_open"]:
# run_job が優先プローブを claim した子プロセスだけは circuit を素通りして実 claude を叩く。
# 結果は record_claude_result に入るので、上限文なら circuit が延び、成功なら閉じる。
if os.environ.get("CLAUDE_QUOTA_PRIORITY_PROBE") != "1":
print(
"CLAUDE_QUOTA_CIRCUIT_OPEN "
f"reason={status['reason']} remaining={status['remaining_seconds']}s",
file=sys.stderr,
)
return EXIT_CIRCUIT_OPEN
print(
"CLAUDE_QUOTA_PRIORITY_PROBE_PASS "
f"reason={status['reason']} remaining={status['remaining_seconds']}s",
file=sys.stderr,
)
(claude-quota-guard.py:416-432)
If you forget to propagate the environment variable, run_job wins the attempt, but the inner run_claude checks the circuit again and immediately returns EXIT_CIRCUIT_OPEN. The one line that gets you through both layers of the gate is env["CLAUDE_QUOTA_PRIORITY_PROBE"] = "1".
And the result of the actual call flows into record_claude_result as usual. If the limit message still comes back, open_until is extended; if the call succeeds, the circuit closes. Whether the probe fails or succeeds, that single result directly determines the circuit's next state — which is what makes this "one real request let through."
Pitfall 1: "one every 30 minutes" is a counter shared by all priority jobs
As the docstring on claim_priority_probe says, PRIORITY_PROBE_INTERVAL is shared across all jobs, not tracked per job. Even if several --priority jobs are scheduled inside the same 30-minute window, the moment the first one passes claim_priority_probe(), last_priority_probe is updated. Every subsequent job evaluates to False at priority and claim_priority_probe() and falls through to SKIPPED without ever touching the real client.
This is intentional. While the limit is genuinely in effect, hammering it with multiple requests won't bring the quota back, so the design throttles the number of attempts themselves. Operationally, though, if you forget that "having multiple priority jobs does not mean each gets its own 30-minute opportunity," you will lose time wondering "why does this one job never get a turn to verify recovery?"
Pitfall 2: when the probe fails, it silently lands as RAN exit≠0
This is the main subject of this post. After subprocess.run, run_job checks whether the probe came up empty and emits a different marker accordingly.
try:
result = subprocess.run(command, env=env, check=False)
except OSError as exc:
print(f"claude quota guard job={label}: {exc}", file=sys.stderr)
return 127
if probe_claimed and result.returncode != 0 and circuit_status()["is_open"]:
# 優先プローブが上限のまま空振りした。RAN exit≠0 のまま残すと quota-catchup.py
# （最新マーカー=SKIPPED だけを再実行）から漏れ、復帰後も当日分が欠番になる
# （2026-09-13〜16 の note2-daily / codex-note-funnel 実測）。SKIPPED として記録する。
print(
"CLAUDE_QUOTA_JOB_SKIPPED "
f"job={label} reason=priority-probe-quota exit={result.returncode} ts={now()}",
file=sys.stderr,
)
return result.returncode
print(
f"CLAUDE_QUOTA_JOB_RAN job={label} exit={result.returncode} ts={now()}",
file=sys.stderr,
)
return result.returncode
(claude-quota-guard.py:499-518)
Before this branch existed, the code only looked at the fact that the probe had won the attempt and actually launched a child process, and fell straight through to the trailing CLAUDE_QUOTA_JOB_RAN. Even when the probe hit the limit again and ended with exit≠0, what remained in the log was CLAUDE_QUOTA_JOB_RAN job=... exit=1.
The problem is that this RAN marker means "already done" as far as quota-catchup.py is concerned. Here is latest_job_marker, which quota-catchup.py uses to narrow down re-run candidates:
def latest_job_marker(paths: list[Path], label: str) -> Optional[Tuple[str, int]]:
"""Return the newest timestamped skip/run marker for one launchd label."""
marker_pattern = re.compile(
r"CLAUDE_QUOTA_JOB_(SKIPPED|RAN)\s+job="
+ re.escape(label)
+ r"(?=\s|$).*\bts=(\d+)(?=\s|$)"
)
...
def latest_marker_is_today_skip(paths: list[Path], label: str, today: datetime.date) -> bool:
marker = latest_job_marker(paths, label)
if marker is None:
return False
marker_type, timestamp = marker
return marker_type == "SKIPPED" and datetime.fromtimestamp(timestamp).astimezone().date() == today
(quota-catchup.py:160-193)
As you can see, this check only looks at the marker type (SKIPPED or RAN) and never inspects the exit code on the RAN side. Even if the probe failed with exit=1, as long as the last log line is CLAUDE_QUOTA_JOB_RAN, latest_marker_is_today_skip returns False and the job quietly drops out of find_candidates' re-run set.
The consequence: even after the circuit really closes, the job sits there with the wrong record — "already RAN today" — until its next scheduled time (the following morning, for instance). The comment records the real-world impact: note2-daily and codex-note-funnel both lost their daily run through this path between 2026-09-13 and 16.
The fix is simple. Check the condition "the probe came up empty while the circuit was open" (probe_claimed and result.returncode != 0 and circuit_status()["is_open"]) first, and only in that case emit CLAUDE_QUOTA_JOB_SKIPPED reason=priority-probe-quota instead of CLAUDE_QUOTA_JOB_RAN. The function's return value stays result.returncode, unchanged. launchd's LastExitStatus still records the actual failure correctly, while only the log marker that quota-catchup.py reads gets relabeled as "still needs a retry." That's the separation of concerns.
Note: The exit code and "should this be retried?" are separate axes. This bug happened because the two had been crammed into a single
RANmarker, and thequota-catchup.pyside never anticipated theexit≠0case. When designing log markers, it's safer not to let "what actually happened" and "what the downstream batch should do next" share the same string.
Summary
- The circuit stops everything uniformly, but jobs that can't recover a missed opportunity — like posting — can hold one attempt every 30 minutes via
--priority - The attempt right is managed by
claim_priority_probethroughlast_priority_probeinlocked_state, and it is shared across all priority jobs, not per job. If multiple priority jobs land in the same window, there's only one opportunity - The child process that wins the attempt must receive
CLAUDE_QUOTA_PRIORITY_PROBE=1. Forget it, and the innerrun_clauderejects the call at the second circuit check - When the probe comes up empty, record it as
CLAUDE_QUOTA_JOB_SKIPPED reason=priority-probe-quota, notCLAUDE_QUOTA_JOB_RAN exit≠0. The re-run check inquota-catchup.pyonly looks at the marker type and never at theexitcode - Design log markers so that "what actually happened" and "what the downstream batch should do" are kept separate
Next time, I'll cover how quota-catchup.py picks up these SKIPPED markers and decides how much to re-run after recovery.
If you run a circuit breaker in front of your own scheduled jobs: does your downstream retry logic distinguish "ran and failed" from "never really got a chance"?
Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*
Top comments (0)