If someone else builds your automations, the most useful thing you can ask for is not documentation. It is a log line you can read. When we work as an agency we agree a logging contract up front, because AI automation agency builds you can debug later start with a predictable record of every run.
The contract
Every run of every automation writes exactly one JSON line when it finishes:
{"flow": "invoice-intake", "run_id": "2026-10-28T09:14:03Z-7f3a", "status": "ok",
"input_ref": "email:msg-8812", "output_ref": "sheet:row-412", "human_review": false,
"duration_ms": 1840, "error": null}
Rules:
-
flowandrun_idare always present. -
statusis one ofok,skipped,needs_review,failed. -
input_refpoints at the source item so a person can find it. -
erroris null unlessstatusisfailed. - No customer message text or personal data in the line, only references.
A checker you can run daily
import json, sys
from collections import Counter
REQUIRED = {"flow", "run_id", "status", "input_ref", "output_ref", "human_review", "error"}
STATUSES = {"ok", "skipped", "needs_review", "failed"}
def check(path):
bad, counts = [], Counter()
with open(path) as f:
for n, line in enumerate(f, 1):
try:
rec = json.loads(line)
except json.JSONDecodeError:
bad.append((n, "not json")); continue
missing = REQUIRED - rec.keys()
if missing:
bad.append((n, f"missing {sorted(missing)}"))
if rec.get("status") not in STATUSES:
bad.append((n, f"bad status {rec.get('status')}"))
if rec.get("status") == "failed" and not rec.get("error"):
bad.append((n, "failed without error"))
counts[(rec.get("flow"), rec.get("status"))] += 1
return bad, counts
if __name__ == "__main__":
bad, counts = check(sys.argv[1])
for (flow, status), c in sorted(counts.items(), key=str):
print(f"{flow:20} {status:13} {c}")
for n, msg in bad[:20]:
print(f"line {n}: {msg}")
sys.exit(1 if bad else 0)
Why this belongs in the agreement
- You can answer "did it run yesterday?" without asking anyone.
- A count of
needs_reviewper flow tells you whether human approval is still needed. - If you ever change agencies, the next team starts from the same record.
Put the contract in the statement of work, and ask for a sample log file before the build ends. More engineering notes live on the NxFlowAI engineering notes.
Top comments (0)