I run a small, mostly automated publishing setup: a scheduled job writes to X through the official API, digital products sit on Gumroad, and a few overnight scripts keep the books. None of it is big. All of it can embarrass me, or cost me money, while I am asleep.
This is the list I now run before any agent, bot or scheduled pipeline is allowed to act without me watching. Every item is here because of something that actually happened. No hypotheticals, no invented incidents.
1. It checks who it is before the first write
I have more than one account logged in on the same machine. So the posting script asks the API "who am I?" before every send and compares the answer with one hard-coded handle. If they differ, it stops. It does not warn and continue.
This costs one extra API read per day (I cache it). It is the cheapest insurance in the whole setup.
2. Old pipelines are assumed armed until proven otherwise
When I rebuilt the setup, I found an older pipeline that was still configured to post for real. It had not posted only because its quality filter happened to reject everything. That was luck, not a safety design.
Now "off" means a visible change: a removed flag, a dated backup of the old file, and a note in the ledger saying what was disarmed and when.
3. An unknown result blocks the next send
Every write records an "attempt" before it goes out and a result after: sent, failed or unknown. A timeout is "unknown", and unknown blocks the queue until a human looks. There is no automatic retry, because a retried post, email or payment is a duplicate.
This is the same idea as the line I keep repeating: treat a 200 as a claim, not a confirmation. Where it matters, read the object back.
4. The spend cap lives at the provider
My scripts have a daily cap of five posts and log the cost of every call. But the cap I actually trust is the one set in the provider's console, because code caps fail exactly when the code is wrong. The script also estimates the remaining balance and warns me about a week before it runs out, not after.
5. I bought my own product and opened the file
This one stung. I did a test purchase of one of my own digital products and opened the ZIP the way a buyer would. About half of it was my internal material: launch plans and notes that never belonged in a customer download.
The fix was a clean build containing only buyer-facing files, then another test purchase to confirm what arrives. Builds are now reproducible, so the same inputs always give the same SHA-256 and I can say exactly which file a buyer received.
6. It ran once under the real scheduler, at night
An unattended job died overnight with a UnicodeEncodeError. The Windows console code page (cp932) could not print a non-ASCII character in a log line. It had worked every time I ran it by hand.
A job is not tested until it has run under the same scheduler, user, locale and encoding it will use in production, and its failure reaches a person instead of a log file.
7. Automation solved posting, not reach
A previous fully automated X setup published 33 posts. The median impressions were about 5. The automation worked perfectly; nobody saw the output.
Since then, I measure replies separately from posts. In my current numbers, short replies on relevant threads get several times the impressions of my own scheduled posts. Agents can draft and schedule. Being part of a conversation still takes a person.
The rule of thumb
If any of these is a "no", the agent runs in dry-run mode until it is a "yes". It is slower, and it is far cheaper than cleaning up after something that acted on its own.
I turned the full list into a one-page checklist with 25 checks across 7 areas. It is free: Agent Ops Pre-Flight Checklist.
I write here weekly about running AI agents and automations in production: what broke, what I check now, and what it cost. Not financial, legal or security advice. Adapt everything to your own systems.
Originally published on NeuralAlpha. I write weekly about running AI agents in production.
Top comments (1)
Point 3 is the one we keep relearning from the other side. I'm on the Auten team (we let AI agents click and type on a real screen), and there a "200" doesn't even exist: the click goes through either way, and the only way to know it worked is to read the screen again afterwards. Is the new row there, did the dialog actually close? Treating "clicked" as a claim and re-reading the state is most of what makes those runs safe to leave alone.
Point 7 matches what we see too: our own scheduled posts reach almost nobody, real threads are where people actually answer.
A question on the unknown state: when a post times out and blocks the queue, how does the human resolve it? By looking up the account's recent posts for that text, or do you store something in the attempt record that you can match later?