DEV Community

Cover image for I let an agent run unattended overnight. At 3am it emailed 400 customers the wrong thing.
Info Inlet
Info Inlet

Posted on

I let an agent run unattended overnight. At 3am it emailed 400 customers the wrong thing.

I found out at 9am what my software did at 3am. That's the whole story of unattended automation in one sentence, and I had to live it before I believed it.

The automation was supposed to be the responsible kind. Every night it reconciled the day's payments against our processor, found the charges that had actually failed, and emailed those customers a polite "your payment didn't go through, here's how to fix it" note before their account lapsed. Dull, necessary, repetitive work — the exact shape of thing everyone tells you to automate. So I automated it. Set it to run at 3am when the processor's data had settled. Set it, and — the phrase that should make every one of us flinch now — forgot it.

One night the reconciliation ran against a half-written ledger. A batch of payments had been acknowledged by the processor's webhook before our own row had actually persisted — the save landed a few seconds late, and the 3am job read the gap as "these payments failed." It did exactly what I built it to do. Tirelessly, correctly, on schedule, it emailed four hundred paying customers to tell them their payment had failed when it hadn't. No human saw the batch. No human saw the send. I saw the support queue at 9am, already two hundred tickets deep, from people who'd paid us and been told by our own software that they hadn't.

Nothing about that automation was broken. Every line did its job. The thing that failed wasn't the code. It was the decision to let the one step that reached four hundred real people happen with nobody in the room.

"Set it and forget it" quietly swaps two different things

Here's the sleight of hand in that phrase, and it took me a 9am support fire to see it.

A recurring job is really two things stapled together. There's the labor — pull the data, match the rows, reconcile the numbers, draft the message, assemble the report. Tedious, mechanical, the same every night, genuinely a waste of a human's evening. And there's the judgment — the half-second where someone looks at the output and goes "wait, two hundred of these are for customers who definitely paid, something's off, don't send." One of those two things is what makes the work a chore. The other is the only thing that was ever protecting you.

"Set it and forget it" sells you on automating the first and quietly automates the second along with it. You think you're deleting the tedium. You're also deleting the one glance that would have caught the half-written ledger. They came stapled together — the person doing the boring reconciliation was also, for free, the person who'd notice the batch looked insane — and the automation rips them apart and throws both away.

That's the move. Nobody decides "I'll remove the human judgment from my payment emails." You decide "I'll stop spending my evenings on reconciliation," which is completely reasonable, and the judgment leaves in the same box as the tedium because you never noticed they were two different things.

The labor was never the dangerous part of the job. The judgment was the whole point of a human being there, and it left in the same box as the tedium.

Unattended doesn't mean fewer failure modes. It means the same ones, at 3am, with nobody there

The seductive story about a scheduled agent is that it's safer — no tired human at midnight fat-fingering a number, no forgotten Monday task, no "oh no I never sent the invoices." And that's real. Automation genuinely removes a whole class of human error.

What it does not remove is the other class: the input that's subtly wrong, the state that's half-written, the edge case the schedule happens to land on. A human running the reconciliation at their desk at 2pm hits the exact same half-written ledger — and stops, because a person looking at four hundred "your payment failed" emails for customers they recognize feels the wrongness before they feel the logic. The agent at 3am hits the same bad state and has no such flinch. It doesn't pause. It doesn't feel anything. It ships.

So unattended automation doesn't give you a system with fewer ways to fail. It gives you a system with the same ways to fail, running at the hour when the fewest people are watching, aimed at the most real-world-facing step, with the one safeguard — a human who'd go "that can't be right" — specifically removed. We tell ourselves we're buying reliability. We're buying the same risk with the smoke detector taken down.

The irreversible step is the one you were never trying to automate

Walk back through my 3am job and separate what I actually wanted off my plate from what I accidentally handed over.

The pulling, the matching, the reconciling, the drafting — take it, please, I never wanted to do any of that at my desk. But the send — the four hundred emails leaving our servers and landing in four hundred inboxes — that was never the part I was trying to get rid of. That part is trivial. It's one button. It takes a human half a second. I didn't automate the send because the send was hard work. I automated it because it was attached to the work I wanted gone, and I let the whole chain run end to end without ever asking whether the last link belonged in the chain.

And the last link is always the dangerous one. It's the step that touches the real world and can't be pulled back: the message that goes out, the payment that moves, the deploy that goes live, the record that gets deleted. Everything before it is reversible — a bad draft is nothing, a wrong reconciliation is a number you recompute. The moment it leaves the building, it's a fact in four hundred other people's lives. That's the step, and it's the one step that costs a human almost nothing to own, because owning it is one glance and one tap.

The tragedy of "set it and forget it" is that it automates exactly the wrong half. It takes the cheap, safe, reversible labor — fine — and then, because it came in the same box, it also takes the one expensive, irreversible step that was the entire reason a person needed to be in the loop at all.

To be clear — this is not "don't automate"

I want to be careful, because the lazy version of this is "automation is dangerous, do it all by hand," and that's how you end up a person manually reconciling payments at midnight, which is both miserable and more error-prone, not less.

Automate aggressively. I do. The recurring report should build itself. The data should pull itself. The draft should write itself. The reconciliation should run every night at 3am exactly as mine did — I would never go back to doing that by hand, and a team that refuses to automate recurring labor out of fear isn't careful, it's just slow and tired. The labor should run while you sleep. That part of "set it and forget it" is a gift.

The claim is narrower and, I think, much harder to argue with: the recurring work and the irreversible step are two different jobs, and the schedule should only own one of them. Let the agent do everything all night. Let it do the whole tedious chain unattended. And then, at the one step that reaches the real world and can't be undone, let it stop and leave that decision sitting on your desk for the morning — not fire it into four hundred inboxes while you sleep. That's not un-automating. It's automating the commute and keeping the signature.

What I do now, since I learned it the expensive way

The 3am email fire changed how I build every recurring thing:

  • I split every automation at the irreversible line. Everything up to and including "assemble the thing that's about to go out" runs unattended on schedule. The step that sends, pays, deploys, or deletes does not fire automatically — it lands in a queue as "here's what I'm about to do to the real world, approve?" The labor runs at 3am. The decision waits for 9am.
  • I make the agent show its blast radius. The approval doesn't say "send emails?" It says "send 400 emails, here are three of them, here's the ledger query they're based on." That one number — 400 — is the entire save. A half-written ledger produces a batch that looks visibly, obviously wrong the second a human sees the count, and a human sees the count before a single email leaves.
  • I stopped trusting "it ran clean" as "it ran right." My 3am job ran perfectly clean every single night, including the night it torched our support queue. Green is not the same as correct, and an unattended system has no one to tell the difference. So the only thing I let run truly unattended is the work that's reversible if it's wrong.

Why this is the exact reason I build the way I do

One level up, it's the same problem, and it's why I build an agent platform the way I do.

Everyone selling autonomous agents right now is selling the 3am dream: it runs while you sleep, it does the whole job, you wake up and it's done. And the demo always stops one frame before the part where it sends four hundred emails to real people on a bad night with nobody watching. The impressive part of an agent — that it can run the whole chain unattended, tirelessly, on a schedule — is also the exact mechanism by which it does something irreversible to the real world before anyone can flinch.

So I never let the schedule own the irreversible step. An agent can run the entire recurring job overnight — reconcile, draft, build, assemble — as an author that produces, tirelessly, at 3am. A second skeptic agent tries to break the output rather than admire it, so a half-written ledger gets caught by something whose whole job is doubt. And every step that reaches the real world — the send, the payment, the deploy, the delete — stops and waits for a human on the button, who sees the blast radius the schedule never will. The labor runs while you sleep. The signature waits for you to wake up. That's the shape of xenition, and it's the same lesson the support queue taught me at 9am: automating the work was always fine. Automating the one decision that reaches four hundred people was never the thing I actually wanted.

I set it and I forgot it. The software didn't forget. It did exactly what I told it, to four hundred people, at 3am, with no one in the room. The labor was never the thing that needed me there.

Top comments (1)

Collapse
 
suppdevbot profile image
DEV SUPPORTS •

You need to verify your account.

Enter fullscreen mode Exit fullscreen mode

tr.ee/dev-to