Silent Failures in Unattended AI Agents — How I Built a Postmortem Tool to Find Them

Chikoku_NEKO — Tue, 11 Aug 2026 17:43:46 +0000

A cron-scheduled agent of mine ran fine: exit code 0, no stderr. But the weekly report it was supposed to write never appeared, and nothing told me. I only noticed days later, checking the directory by hand.

I call this a silent failure, and I think it's the nastiest failure mode of unattended agents: the job reports success, so no alerting ever fires, and the only detection mechanism left is a human happening to look. I built agent-coroner to catch these automatically. v0.1.0 is now on GitHub, PyPI, and the Claude Code plugin marketplace.

What happened

For five weeks straight, my weekly-report job ran without a problem. It pulled data, generated markdown, wrote a file every Friday morning. I didn't monitor it actively, because it was boring infrastructure that appeared to be working.

Week six: no report. The logs showed nothing obviously wrong, and the job had no alerting of its own. By week seven it had recovered on its own and everything looked normal again. To this day I haven't root-caused that one missing week. That's the part that bothered me most: without an artifact check, I wouldn't even have known it happened.

The design

Instead of trying to guess when things go wrong, I decided to be explicit about what each job should produce, then check those promises deterministically. There are three layers.

The first layer is a contracts file, where you declare what each job promises:

jobs:
  weekly-report:
    schedule: "FRI 08:30"
    grace_minutes: 60
    artifacts:
      - path: "reports/weekly-*.md"
        max_age_hours: 192
        min_bytes: 500
    logs: "run.log"

The second layer is the checker, a small CLI you run from cron or Task Scheduler. It asks four questions: does the file exist, is it fresh (max_age_hours), is it big enough (min_bytes), and did your optional verify command pass. No LLM is involved, so a run takes seconds and ongoing monitoring costs nothing. It exits non-zero if a contract is violated and zero otherwise, so you can wire it into any alerting you already have.

The third layer is the autopsy. Only when a contract breaks, Claude is dispatched read-only (--allowedTools "Read,Grep,Glob") to read the logs and write a four-section postmortem: facts, hypotheses ranked by confidence with log quotes, verification steps for a human to run, and prevention suggestions. It diagnoses; it never modifies anything.

One design decision worth spelling out: detection never depends on the LLM. The checker always runs first, and if the autopsy crashes, the violation notification still goes out. There's also no self-healing on purpose. Once you let an LLM patch things in an unattended system, failures start getting hidden instead of reported, and I trust a monitor that only reports.

Dogfooding it on a new project

The day after publishing, I started a new project (a paper-trading eval that forecasts day-ahead prices on JEPX, Japan's electricity exchange) and registered its weekly accuracy report as a coroner contract. Just building and registering that pipeline surfaced two traps of exactly the kind the tool exists for.

The first trap: the JEPX CSV endpoint returns HTTP 200 with a zero-byte body when the Referer header is missing. Success code, no data. I caught it with curl while writing the fetch layer. A status-code-only check would have "succeeded", saved an empty file, and quietly poisoned everything downstream. The defense is now two layers deep: the fetcher rejects empty bodies outright, and the contract's min_bytes rule guards the artifact. If one goes quiet, the other still fires.

The second trap: my monitored artifacts live in a different repository on a different drive, so I wrote the contract with an absolute-path glob, and the checker crashed. Glob resolution is delegated to pathlib's relative globbing, so absolute paths don't work. The fix is to set workdir (which defaults to the directory containing contracts.yaml) and use a relative glob. After rewriting it, the checker passed: power-eval-weekly: ok, exit 0.

I found the second one funny in hindsight. Putting the verification tool into real service exposed a gap in the verification tool's own docs, so the verifier got verified. The README now documents workdir and the relative-glob rule.

Quick start

Install:

uv tool install agent-coroner   # or: pip install agent-coroner

The claude CLI on PATH is required only for autopsies; the checker works standalone.

Write contracts.yaml:

jobs:
  weekly-report:
    schedule: "FRI 08:30"
    grace_minutes: 60
    artifacts:
      - path: "reports/weekly-*.md"
        max_age_hours: 192
        min_bytes: 500
    logs: "run.log"

Run the checker:

$ coroner check --config contracts.yaml --no-autopsy
[coroner] 1 violation detected
- weekly-report / reports/weekly-*.md: missing (no file matches 'reports/weekly-*.md' under <contracts-dir>)
$ echo $?
1

Once reports/weekly-*.md exists, is fresh, and meets min_bytes:

$ coroner check --config contracts.yaml --no-autopsy
weekly-report: ok
$ echo $?
0

*/15 * * * * coroner check --config /path/to/contracts.yaml >> /var/log/coroner.log 2>&1

On Windows:

schtasks /create /tn "coroner-check" /tr "coroner check --config C:\path\to\contracts.yaml" /sc daily /st 09:00

Notifications are delegated to an external command of your choice (email, Slack, whatever), with {message_file} holding the path to the full message text.

Claude Code integration

If you use Claude Code, there's also a plugin:

/plugin marketplace add Chikoku-NEKO/agent-coroner
/plugin install agent-coroner@agent-coroner-marketplace

It adds /coroner-status (which jobs are ok, which have violations), /autopsy <job-name> (run a postmortem on demand), and a SessionStart hook that surfaces unread postmortems when you open a session.

Closing

If you run anything unattended that's supposed to produce an artifact — weekly reports, data pipelines, scheduled Claude jobs — an existence-and-freshness check on the artifact is cheap insurance, whether or not you use this tool. agent-coroner just makes it declarative: one YAML file, a deterministic checker, and a read-only postmortem when something breaks.

Links:

GitHub: https://github.com/Chikoku-NEKO/agent-coroner
PyPI: https://pypi.org/project/agent-coroner/
Claude Code marketplace: agent-coroner plugin

If you try it and something breaks during registration, file an issue. That's how the two traps above got documented.