My Slack bot was firing success messages while D1 row count sat completely frozen. The agent wasn't crashing — it was completing its Slack notification step and skipping all the actual work. Alerts alone would never have caught this.
The fix was treating Slack, D1 logging, and cost tracking as one connected pipeline instead of three separate tools. The schema I landed on is deliberately lean — 7 columns, with tokens_used being the one that matters most:
CREATE TABLE agent_runs (
id TEXT PRIMARY KEY,
agent_name TEXT NOT NULL,
run_at INTEGER NOT NULL,
status TEXT NOT NULL, -- 'ok' | 'error' | 'timeout'
duration_ms INTEGER,
tokens_used INTEGER,
error_msg TEXT
);
Without tokens_used in D1, your cost dashboard only tells you what you spent today — not which agent burned it. I was mentally estimating my analysis tool's monthly Claude spend at around $20. The actual number was $38. I had no idea until I ran the aggregation query against the logs.
The other thing that bit me: MCP stdio transport timeouts. I spent days convinced it was an OAuth misconfiguration. It wasn't. One tool was calling an external API that occasionally took 30+ seconds. The stdio transport's read timeout was shorter than that, and Workers' 30-second wall clock limit made it worse — the subprocess got force-killed with a misleading exit code 1 error. The timeout status rows piling up in D1 are what actually surfaced the pattern. Without that log, it would have stayed a vague "sometimes slow" complaint.
On the Workers side: if you fire Slack notifications with a plain await instead of wrapping in ctx.waitUntil, the fetch gets killed the moment your Worker returns a response. That's why Webhooks seem to fail randomly — they're not failing, they're being terminated. Also worth knowing: Slack returns a 400 if your payload exceeds 4,000 characters, and raw error stack traces will blow past that limit silently.
I wrote up the full breakdown — including the KV write explosion that hit 12K writes in 2 minutes and how I traced it back to a specific agent using D1, plus the Durable Object refactor for slow tools — over on riversealab.com.
Top comments (2)
This is a perfect example of why success messages should be downstream of the durable state change, not parallel to it. If Slack says done but the database is unchanged, the notification is just theater. I like that you monitored the workflow from the artifact outward instead of trusting the agent's last sentence.
The line that lands is "the agent wasn't crashing — it completed its Slack notification step and skipped the actual work." The success was real, but it belonged to the wrong thing: the notification fired, the row count didn't move.
We hit the same split from the other side. Your stack makes the mismatch visible from durable state — timeout/status rows pile up in D1 and the pattern shows in aggregate. What helped us as a complement was binding a claim to its artifact at the moment it's made: a step that reports "done" has to point at what changed — the row delta, the file mtime, the test result. If that pointer is empty, the "done" stays visibly second-class instead of being accepted as a completion claim.
So I read the two approaches as different gates: your logging catches the system-level pattern over time; claim-time receipts catch the local "done with no artifact" moment before it becomes trusted by the operator.
Where would you draw the line between the two? The retry/timeout rows you mentioned feel like they'd slip past a claim-time check — that's exactly the band where log-and-aggregate seems irreplaceable.