DEV Community

Cover image for I gave my AI agent a safety rule. It quietly stopped shipping for four days.
dustin chu
dustin chu

Posted on Originally published at wisplu.com

I gave my AI agent a safety rule. It quietly stopped shipping for four days.

On 22 August at 00:46, my autonomous ops agent finished its first article, then stopped and
asked me to approve it — exactly as instructed.

I got to it on the morning of 24 August. Nothing shipped for four days.

The four days are not the interesting part. The interesting part is that
the agent looked productive the entire time, which is precisely why the problem was
invisible until it wasn't.

The rule, and why it seemed obviously correct

The agent runs on Claude Code. Every few hours it wakes up, reads its state files, pulls one
task off a queue, does it, and writes back what happened. Its operating rules included this:

Do not publish the first article. Post the draft path and wait for approval.

Rationale: the expensive failure mode for an unattended writer isn't one bad article —
it's twenty articles in the same systematically wrong voice.

That reasoning is sound. When something produces continuously without supervision, the
risk that actually hurts is a repeated error, not an occasional one. A single human checkpoint
is cheap insurance.

The same rules also anticipated the obvious failure:

When you need a human, don't idle. Mark the task blocked and pick up the next
executable task
.

Checkpoint for quality, bypass for throughput. On paper, both failure modes were covered.

What actually happened

The agent followed both rules faithfully. Straight from its log:

Wake What it did
08-22 00:46 First draft finished. Posted for approval.
08-22 08:45 Instead: submitted the sitemap to Search Console
08-22 16:48 Instead: fixed external images across 41 archived posts
08-23 00:46 Instead: wrote the second article
08-23 08:45 Instead: built a metrics collection script
08-23 16:45 Instead: generated social share cards
08-24 08:48 Instead: fixed layout shift on article pages
08-24 09:30 Out of work

Eight wakes. Seven completed tasks. Zero published.

Look at the last row. That isn't a crash. That's the agent finishing every task in the
queue that didn't require a human. Everything left was waiting on someone to answer something.

Why this class of failure hides so well

Had the agent stalled on day one, I'd have noticed on day one.

It didn't. It fixed images, wrote scripts, generated assets — every one a real improvement,
every log entry marked "done."
Judging by output volume, those four days were
indistinguishable from the four before them.

The "don't idle" rule saved the agent and hid the problem at the same time. The pipeline
had stopped, but from the outside it looked busy.

And fallback work is finite. It gets consumed. The moment it runs out is the first time the
system can honestly report that it stalled — four days after it actually did.

My first diagnosis was wrong

My instinct was: the checkpoint is too strict, loosen it.

That's wrong in a specific and common way — it blames the rule, and the rule is a symptom.

The actual problem has two layers.

Layer one: I priced the checkpoint against the wrong risk.
It was insurance against twenty badly-voiced articles reaching readers. How many readers did
the site have at that point? Zero. On a site with no audience, publishing a bad article
costs approximately nothing and is trivially reversible. I applied a caution that only makes
sense once you have readers to a stage that had none.
The checkpoint wasn't wrong; it was
installed too early.

Layer two, the real cause: the raw material for the writing only existed in my head.

The articles were supposed to be first-hand — things I'd personally hit, processes I'd
personally run. Which means every article required my availability. The approval gate
merely surfaced that dependency early. Without the gate, the agent would have stalled by
article three asking for material.

I built a pipeline and connected its input to the busiest person in the company.
That's not an execution problem. That's a design error.

The fix was the subject matter, not the rule

So I didn't loosen the checkpoint. I changed what the agent writes about.

The subject moved from "things that need my experience" to "this agent's own operating
record"
— which is what you're reading. The raw material now comes from the system itself.
It generates records continuously, including records of its own failures, and nobody has to
approve anything for that supply to exist.

The checkpoint stayed. I just moved it to take effect once there are actually readers.
A safeguard should arrive at the same time as the thing it safeguards. Installed early, it's
just a roadblock.

If you're building one of these

1. Treat every "needs a human" step as a single point of failure.
For each approval you add, ask: if this person is unavailable for a week, what happens?
If the answer is "the pipeline stops," it isn't a process step — it's a dependency that can
sever the whole chain, and everything upstream of it stops mattering.

2. Track how many tasks remain that need no human.
That number is your actual runway. When it hits zero, you've stalled — and it will hit zero
sooner than you expect. Mine lasted four days.

3. Before choosing what to work on, ask where the input comes from.
This step gets skipped most and costs most. However good the subject, if the first-hand
material lives in one person's head, the pipeline's ceiling is that person's spare time.


One last thing. Every log entry across those four days said "done."

Reading the logs would never have revealed that the system had stopped. What surfaced the
problem was running out of fallback work — resource exhaustion, not monitoring.

So if your automated system has never once told you "I have nothing left to do," that may not
mean it always has work. It may mean you never gave it a way to say so.

Top comments (0)