When an outside team hands over an automation, the client usually gets a runbook and a demo. What they rarely get is a tiny script that answers one question every morning: is it still working?
This is the pattern we leave behind as an AI automation agency that hands over checks, not just code. Adapt the checks to your own stack.
The five checks
- The last successful run happened recently.
- No run is stuck in "processing".
- The error log is quiet since yesterday.
- Credentials are still valid (a cheap read call succeeds).
- The approval queue is not piling up.
import json, sys, datetime as dt
def load(path):
with open(path) as f:
return [json.loads(line) for line in f if line.strip()]
def smoke(runs, approvals, max_gap_hours=6, max_pending=20):
now = dt.datetime.utcnow()
problems = []
ok = [r for r in runs if r["status"] == "ok"]
if not ok:
problems.append("no successful run on record")
else:
last = max(dt.datetime.fromisoformat(r["ts"]) for r in ok)
if (now - last).total_seconds() > max_gap_hours * 3600:
problems.append(f"last success at {last.isoformat()}")
stuck = [r for r in runs if r["status"] == "processing"]
if stuck:
problems.append(f"{len(stuck)} runs stuck in processing")
errors = [r for r in runs if r["status"] == "error"]
if errors:
problems.append(f"{len(errors)} errors in log")
if len(approvals) > max_pending:
problems.append(f"{len(approvals)} items waiting for approval")
return problems
if __name__ == "__main__":
issues = smoke(load("runs.jsonl"), load("approvals.jsonl"))
print("OK" if not issues else "\n".join(issues))
sys.exit(1 if issues else 0)
Check 4 depends on your tools, so add one cheap read call per integration.
Where to run it
A scheduled job that emails the owner only when the script exits non-zero. Silence means healthy.
For the thinking behind choosing what to automate at all, see our blog on workflow audits and custom builds.
Top comments (0)