It Ran Every Morning and Still Broke: Failure Modes of a Free-Tier AI Job

The cron log said exit 0 every morning. The summary file updated on schedule. The job was running. The output was wrong.

This is an autopsy of a small automation that failed without crashing. It ran on MonkeyCode's free tier: a daily job that fetched upstream release notes, summarized them with a model, and wrote the result to a file. The setup was simple. The failure modes were not.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The setup

The job was deliberately boring. One script, one cron line, three files:

• watch.py — fetch the latest release, build a prompt, call the model, write the summary.
• latest_summary.md — the output a human reads.
• ledger.jsonl — one JSON line per run, recording the date, release tag, and model output.

The goal was not to build an agent. It was to replace a manual habit — reading release notes every morning — with a script that did it at 7:00 AM. The job ran on a free server with a free token allowance. The requirements were simple. The failures were not.

Failure mode 1: the model answered a question nobody asked

The prompt requested three sections: breaking changes, migration steps, and a verdict. One morning the model returned exactly:

Yes.

One word. No sections. The script wrote it to the summary file, appended a ledger entry, and exited zero. A human reading the summary would have no idea the automation had failed. The script had no idea either.

The fix was not prompt engineering. It was a schema check — verify the output has all three sections before writing anything.

Failure mode 2: a quota message inside a 200 response

Later in the trial, the job stopped producing summaries. The script did not raise an exception. The HTTP status was 200. The response body contained a quota message from the model endpoint, and the script treated it as the summary.

This is the failure mode that free tiers make likely: rate limits and quota exhaustion often surface as normal responses, not as errors. The fix is to validate the response body against an expected shape, not just the status code.

Failure mode 3: the cron environment was not your environment

The script worked in a terminal. On the server, it produced empty output with a clean exit. The cause was a PATH difference: the cron environment did not include the directory where a dependency lived.

The fix was two lines at the top of the script: absolute paths and a startup check that verifies every dependency before doing real work.

Failure mode 4: stale state looked like fresh data

The script tracked the last processed release in a state file. One day the upstream API call failed. The script caught the exception and exited zero. The next day the API worked, but the script saw the same release tag in the state file and skipped the work. The summary stayed stale, and every run looked successful.

The fix was ledger discipline: write the ledger entry before updating the state file, and log every skipped run as an event.

Failure mode 5: truncation that looked complete

Release notes are long. The script truncated them to 4,000 characters before sending them to the model. One day the breaking change was in the truncated portion. The model summarized what it saw — a non-breaking release. The summary said "no breaking changes." The release had three.

The fix was a warning log: when truncation happens, record it. You cannot fix what you cannot see.

The health check I wish I had from day one

After the fifth failure, I wrote a single script that catches all five failure modes in one run. It is the artifact this article is really about.

#!/usr/bin/env python3
"""healthcheck.py — verify a free-tier AI job is actually healthy."""
import json, sys
from datetime import date

failures = []

def check(name, ok, detail):
    print(f"[{'OK' if ok else 'FAIL'}] {name}: {detail}")
    if not ok:
        failures.append(name)

# 1. Output is non-empty and non-trivial
with open("latest_summary.md") as f:
    content = f.read()
check("output_size", len(content.strip()) > 50, f"{len(content)} chars")

# 2. Output has the expected structure
check("output_schema", all(s in content for s in ["Breaking", "Migration", "Affects"]),
      "expected three sections")

# 3. Ledger has a recent entry
with open("ledger.jsonl") as f:
    entries = [json.loads(l) for l in f if l.strip()]
last = entries[-1]
check("ledger_entry", last.get("date") == date.today().isoformat(),
      f"last entry: {last.get('date')}")

# 4. State file matches the ledger
with open("last_seen.json") as f:
    state = json.load(f)
check("state_consistency", state.get("tag") == last.get("tag"),
      f"state={state.get('tag')} ledger={last.get('tag')}")

# 5. No truncation warnings in the log
with open("watch.log") as f:
    log = f.read()
check("no_truncation", "TRUNCATED" not in log, "truncation warning found")

sys.exit(1 if failures else 0)

Run this as a second cron job, five minutes after the main job. If it exits nonzero, you have a problem worth waking up for. If it exits zero, the automation is at least honest about what it did.

Who should not use this pattern

If your automation is a script you run by hand, a health check is ceremony. If your automation produces content that goes straight to readers, a health check is the minimum — add a human review step on top. If your logs contain secrets, sending them to a third-party model is a policy decision, not a technical one. And if you need a guaranteed SLA, free infrastructure is the wrong foundation; check the current terms and limits before relying on it.

The lesson

Exit code zero means the script ran. It does not mean the job worked. The difference is the gap this health check closes.

Free tiers are not worse because they fail. They are worse because they fail quietly. Name the failure modes, write the checks, and the quiet failures become loud ones — which is exactly what you want.

MonkeyCode provides free models that can run this workflow.