My Safety Switch Was the String "dry". Python Called It True and Emailed 1,412 Customers.

At 06:58 on a Tuesday my agent printed this:

[06:58:12] boot   | AGENT_MODE=dry | target=plan-metadata | scope=all
[06:58:14] boot   | dry-run guards active
[06:58:19] stripe | updating customer metadata | n=1412
[06:58:51] mail   | queued plan-change notice | n=1412
[06:58:52] done   | 1412 updated, 1412 queued, 0 failed

Read that twice. Line one says dry. Line two says the guards are active. Line three updated fourteen hundred and twelve live Stripe customers. Line four queued the same number of customer emails. Line five calls it a success.

I read that log at 07:02, filed it as my most boring successful run of the week, and made coffee. The queue worker drained at 07:00. By 11:20 I had eleven replies from people asking why they'd been moved to an annual plan, and one very polite email from a customer who wanted to know why my "dry run" had a body count.

Nothing was actually changed in billing. The agent was rewriting a metadata label as part of a segmentation cleanup, and the "notice" email told people their plan details had been updated. That's the good version of the story. The bad version is that the same code path sat three lines from a proration_behavior parameter I'd commented out "for now." I've thought about that a lot since.

The bug was one missing comparison

Here is the entire guard:

import os
import logging

log = logging.getLogger("agent")

MODE = os.getenv("AGENT_MODE", "dry")

def sync_customer(customer, metadata):
    if os.getenv("AGENT_MODE", "dry"):
        do_the_real_thing(customer, metadata)
    else:
        log.info("dry run: would update %s", customer.id)

os.getenv returns a string or None. It never returns a boolean, no matter what you put in .env. And every non-empty string in Python is truthy — including the string "dry", including "false", including "0", including "no". So the condition above isn't "are we in dry mode?" It's "does this variable exist?" Which it did. Which means the guard was never a guard. It was a comment with syntax.

The startup banner didn't lie, exactly: it printed the variable with an f-string and faithfully reported dry. It just reported a value no branch ever compared against anything.

For the record, because I looked it up sitting on my kitchen floor:

Value in .env	bool(value)	Passes if value:
"dry"	True	✅
"false"	True	✅
"0"	True	✅
""	False	❌
unset	None → False	❌

Exactly one string there turns the switch off: the empty one. I'd written a boolean check so fragile that safety required configuring nothing at all.

Why my tests didn't catch it

This is the part I'm least proud of, and it's the part worth reading.

I had one test for dry run:

def test_dry_run_writes_nothing(monkeypatch):
    monkeypatch.setenv("AGENT_MODE", "")   # ← the tell
    sync_customer(fake_customer, {"tier": "grandfathered"})
    assert stripe_mock.update.call_count == 0

Look at what I passed. "". The empty string. I picked the single value that makes the broken condition evaluate the way I wanted, ran it, watched it go green, and never asked why I needed "" to represent "dry mode" when the actual config file said dry.

That test didn't verify the guard; it verified that I could turn the guard off. It passes as long as the software is broken in exactly the way it's broken.

Three other things had to line up for this to reach 1,412 people:

1. The flag was read in two places. MODE was assigned at import and used by nothing, while the functions re-read the environment on every call. Two sources of truth, three call sites, zero agreement.
2. Dry run only covered writes. It never touched the mail queue. Even a correct guard would have left a dry-run message sitting in the outbox table for the worker to pick up at 07:00. The dry run and the live run shared the same outbox.
3. Nothing asserted on startup. I printed the resolved config as a courtesy, in a log nobody reads, with no format that announces when the value is wrong. A startup banner that only ever prints is decoration.

The fix: don't branch on the flag near the side effect

The instinct after an incident like this is to add == "live" everywhere. That fixes the bug and leaves the pattern intact. Bad flags get copied; the pattern is what propagates. Three changes, in increasing order of how much they'd have saved me.

1. Parse config once, into a type, and crash on anything unrecognized.

from enum import Enum

class Mode(str, Enum):
    DRY = "dry"
    LIVE = "live"

def parse_mode(raw: str | None) -> Mode:
    if raw in (Mode.DRY, Mode.LIVE):
        return Mode(raw)
    raise SystemExit(f"AGENT_MODE must be 'dry' or 'live', got {raw!r}")

{raw!r} matters more than it looks. repr puts the quotes back. Half my incident's confusion came from a log line that stripped the one piece of information that would have identified the bug. And an allowlist of two values means there is no third state to guess about: "true", "1", "yes", "off" all become a startup crash instead of a policy decision made by Python's truthiness rules.

2. Enforce at the boundary, not at the decision.

The guard belonged in the object that touches the network, not in the function that decides what it wants to happen:

class StripeWrite:
    def __init__(self, client, mode: Mode, canary: str | None):
        self._client = client
        self._mode = mode
        self._canary = canary
        self.audit: list[dict] = []

    def update_metadata(self, customer_id: str, metadata: dict):
        if self._mode is Mode.DRY:
            self.audit.append({"id": customer_id, "md": metadata})
            return   # never reaches the client
        if self._canary and self._canary not in metadata.values():
            raise RuntimeError("canary tag missing — refusing to write")
        return self._client.customers.update(customer_id, metadata=metadata)

Now every write goes through one door, and the door reads the mode. There's no path to the client that skips the check, because the check sits in front of the client rather than up the call stack where a future refactor can route around it.

3. Make dry run physically incapable, not politely unwilling.

The final version constructs the whole agent with a read-only API key when the mode is dry, and points the mail client at a sink that appends to a file. That's the change I trust. Every other fix depends on my code evaluating a condition correctly. This one depends on the platform rejecting an unauthorized request — a much better thing to depend on.

If your dry run can't fail because of a bug in your code, it isn't a dry run. It's a rehearsal that can go live.

Plus the boring procedural bits that actually held on the second attempt: a canary batch of one record tagged canary:<date>, a rule that no run touches more than 25 records without a human reading the diff, an outbox worker that pauses when ACTIVE=false, and a startup check that refuses to boot when mode is LIVE and canary is None.

What I'd tell anyone running an agent against production

• Grep every os.getenv, os.environ.get, and getenv(...) or default in your agent. If one is used in an if, it's a string pretending to be a boolean and it's probably wrong.
• Test the config parser with the full ugly set: "true", "1", "yes", "on", "false", "0", "no", "off", "", "none", "nil". Then test that every unrecognized value crashes instead of defaulting.
• If a dry run has ever "worked," check the log for evidence that it took a different branch than the live run. Mine printed the same success line in both modes. That should have been impossible.
• Decouple nothing on the write path. Every side effect the live run can produce — emails, webhooks, queue rows, uploads — needs a stubbed twin in dry mode, or it isn't covered.
• Blame the code before the model. I spent forty minutes reading the system prompt looking for a reason the agent "decided" to run live. The agent did exactly what it was told. It was told by four lines of Python I wrote in a hurry.

The humiliating summary: I built a safety system out of a string, then wrote a test that only passed because the string was empty. Both took less time to write than this post. The emails took about forty seconds.

I now have an AGENT_MODE parser with eleven test cases, one write boundary, a read-only key for dry runs, and a startup check that will refuse to boot rather than guess. The agent hasn't changed at all. It was never the agent.

---

I write up the specific playbooks — the config parsing, the write boundaries, the canary rules, the startup assertions — in The Solo Operator's AI Agent Playbook — code LAUNCH90 at checkout makes it $1.90. If it doesn't save you 5 hours in week one, reply to the receipt for a refund.