I have a few scheduled Claude tasks that work while I'm away. One posts to X on a fixed schedule and records the numbers. Another runs a weekly review and writes the next week's drafts. Each run ends with a report in the chat.
A model's prose report is easy to read and hard to add up. The fact you need can sit anywhere: a sales figure in the fourth paragraph, a deadline mentioned in passing. So the reports from these tasks end with a short block of lines like these (example values):
RPT_REVENUE {"v":1,"product":"Example ebook","month":"2026-11","sales_mtd":0,"count_mtd":0,"currency":"USD","as_of":"2026-11-16","source":"store dashboard"}
RPT_EVENT {"kind":"launch","title":"Example article scheduled","date":"2026-11-17","status":"scheduled"}A tag, a space, and one JSON object on one line. The prose stays above it for me. The lines are for anything that needs to count.
Here is how I set the rules, the small parser I'd start with, and a check you can run on your own scheduled jobs in about twenty minutes.
Why one line per result
A report written by a model drifts. The same number shows up as "$0", "zero so far", "no sales yet" or a sentence that never quite says it. That's fine for reading and bad for adding up.
A single line per result fixes the shape:
One record, one line. The JSON is never wrapped across lines, so a plain line-by-line reader can find it.
A tag first. RPT_REVENUE, RPT_EVENT, RPT_RESULT. The tag tells the parser which fields to expect.
Required fields per tag. A revenue line without as_of or source is incomplete, and the parser says so.
At the end of the report. The lines go after the prose. They don't need to go into phone notifications, which should stay short.
The point is that a script can read a week of reports and produce a table, and I can read the table.
The rule that matters most: zero and missing are different
This is the one I'd copy even if you skip everything else.
My rule for every number: if the run actually looked (opened the dashboard, read the figure) and saw 0, it writes 0. If it couldn't look, because the login had expired or a captcha appeared, it writes no line at all, and the prose says "not retrieved" with the reason.
The reason is arithmetic. A 0 that really means "I couldn't check" gets added into the month like any other 0. Weeks later the total looks precise and isn't. A missing line with a "not retrieved" note stays visible: the parser below lists it as a to-do, and I go and look.
Two more rules in the same spirit:
Only what this run confirmed. The material for these lines is the run's own results and what's already written in the status ledger. If it doesn't know, it doesn't write the line. No estimates.
Don't repeat. If an event was already reported in the same state, it isn't reported again. A deadline that is still "scheduled" stays quiet until it changes.
Quiet measurement errors are the other half of this. For its first few days, my metrics script left replies out of the count. After the fix, a measurement on one day showed my five manual replies averaging about 22 impressions and nine automated posts averaging about 7. A format check can't catch a bug like that, because the numbers were well formed. What the source and as_of fields do is tell me which numbers to recheck once I find a bug like that.
A run that never reports
One failure doesn't produce a wrong line. It produces no report at all.
In October my PC restarted overnight for a Windows update, and the scheduled tasks between late evening and the next morning didn't run. Nothing was half-done (the posting ledger had no unknown entries), and I chose not to post the missed slots later. The change was a question each morning run now asks: has it been more than 24 hours since the previous run? If yes, it says so in the report.
A parser can't notice a report that doesn't exist. Something has to compare "when did this last run" with "how often should it run".
A parser to start with
Standard library only. It reads a report (a file or stdin), checks each tagged line and lists anything that needs a person. It doesn't judge whether a number is true, only whether the line is complete and well formed.
!/usr/bin/env python3
"""Check the one-line records at the end of an agent's report.
python3 report_lines.py report.txt
some_command | python3 report_lines.py
"""
import json, re, sys
REQUIRED = {
"RPT_REVENUE": {"product", "month", "sales_mtd", "count_mtd", "currency", "as_of", "source"},
"RPT_EVENT": {"kind", "title", "date", "status"},
"RPT_RESULT": {"task", "status"},
}
TAGGED = re.compile(r"^(RPT_[A-Z]+) ({.})\s$")
def check(text):
records, problems, seen = [], [], set()
for n, line in enumerate(text.splitlines(), 1):
if line.lower().startswith("not retrieved"):
problems.append(f"line {n}: {line.strip()}")
continue
if not line.startswith("RPT_"):
continue
m = TAGGED.match(line)
if not m:
problems.append(f"line {n}: tag without a one-line JSON object")
continue
tag, body = m.groups()
if tag not in REQUIRED:
problems.append(f"line {n}: unknown tag {tag}")
continue
try:
rec = json.loads(body)
except ValueError as e:
problems.append(f"line {n}: bad JSON ({e.msg})")
continue
missing = sorted(REQUIRED[tag] - rec.keys())
empty = sorted(k for k, v in rec.items() if v is None or v == "")
if missing or empty:
problems.append(f"line {n}: missing {missing} empty {empty}; leave the line out rather than guess")
continue
if tag == "RPT_REVENUE" and not isinstance(rec["sales_mtd"], (int, float)):
problems.append(f"line {n}: sales_mtd must be a number, got {rec['sales_mtd']!r}")
continue
key = (tag, json.dumps(rec, sort_keys=True))
if key in seen:
problems.append(f"line {n}: duplicate of an earlier line")
continue
seen.add(key)
records.append((tag, rec))
return records, problems
if name == "main":
text = open(sys.argv[1], encoding="utf-8").read() if len(sys.argv) > 1 else sys.stdin.read()
records, problems = check(text)
for tag, rec in records:
print(f"ok {tag:<12} {rec.get('product') or rec.get('title') or rec.get('task')}")
for p in problems:
print(f"TODO {p}")
sys.exit(1 if problems else 0)Feed it a report that contains a valid revenue line, a line with "sales_mtd": null, and a sentence starting "Not retrieved: ebook store (login expired)". It prints one ok and two TODO lines, and exits with 1 so a wrapper can notice. Add your own tags to REQUIRED as you need them.
The 20-minute check
Pick one scheduled job that reports to you, and the last few reports it produced.
Find the two facts you actually read it for (5 min). Usually a number and a status. Write them down as fields with names.
Write the line format (5 min). One tag per kind of fact, the required fields for each, one JSON object per line. Put the format in the task's instructions, with one example line.
Write the zero rule into the instructions (2 min). "If you saw 0, write 0. If you couldn't check, write no line and say 'not retrieved' with the reason."
Run the parser on the next report (3 min). Fix the instructions until it passes without edits by hand.
Add the missed-run check (5 min). At the start of the job: when did the previous run finish? If the gap is longer than the schedule allows, the first line of the report says so.
Once a few jobs write these lines, you can collect them into one file and let a short script turn it into a table.
What this doesn't solve
The lines are written by the same model that wrote the prose. The parser checks format, so a well-formed wrong number passes. I treat source and as_of as the audit trail: when something looks off, they tell me where to look and when the figure was read. I also keep the rule that each run reports only what it checked during that run, which is easier to verify than a summary of everything it remembers.
If you want the rest of the checks I run before letting a job act on its own, the free one-page checklist is here: https://neuralalpha.gumroad.com/l/agent-ops-preflight?utm_source=substack&utm_medium=article
Originally published on NeuralAlpha. I write weekly about running AI agents in production.
The operating kit I use for this (guide, skills, templates, tested scripts): AI Staff Operating Kit for Claude Code.
Top comments (0)