What actually breaks when you let an LLM execute real actions

I built a control plane that sits between an LLM and anything that costs money
or has side effects — the model proposes an action, hard rules and a business
validator decide what actually runs. I wrote up the architecture
here
a while back; the pattern itself is open source
(SACP, MIT).

This isn't that post. This is the list of things that actually broke — found
by running it against a real database and a real model provider, not by
reading the spec. A few of these got a one-line mention last time; here they
get the full symptom → cause → fix. None of them are exotic. That's the point:
they're the bugs you get for free the moment a non-deterministic component
sits inside a system that has to be deterministic and auditable.

The decision engine

Date.parse() accepts strings that aren't dates

A schema validation test passed with expiresAt: "tomorrow at 3".

V8's Date.parse() accepts strings that aren't ISO 8601 and happily returns a
timestamp instead of NaN. The validator trusted Date.parse() alone to
decide whether a field was a valid date.

const ISO_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/;
const isIso = v => typeof v === 'string' && ISO_RE.test(v) && !Number.isNaN(Date.parse(v));

Never validate ISO format with Date.parse() alone. Regex first.

The policy engine silently allowed everything

14 of 22 integration tests failed — every policy returned allowed: true, for
every input, including the ones that should have been blocked.

The test suite required the engine but never called registerAll(). With a
lazy registry and zero policies actually registered, the engine's default
behavior was to allow. Not a bug in the policy logic — a bug in what happens
when there's no policy logic at all.

If a registry is lazy, the test has to activate it explicitly. And a
fail-open default is a loaded gun: it works fine right up until someone forgets
to load the policies, and then it works silently wrong.

Model IDs copied from a marketing page

400 json_validate_failed on every call to the provider.

The model selector listed IDs copied from a marketing page. They
weren't real, callable model IDs — marketing names and API names aren't
guaranteed to match, and in this case they didn't. Verified against the
provider's live model list and replaced them. Marketing names lie; verify
against the real API before anything goes in the registry.

The model invents an expiry date in the past

decision: allow, but the final state was blocked, with
businessValidationResult.failures = [DECISION_EXPIRED].

The model generated a date from somewhere near its training cutoff, not from
the actual moment of the call — it doesn't know what time it is. The business
validator did exactly its job and blocked an already-expired decision. The fix
is two-sided: normalize server-side after schema validation, and tell the
model the real time in the system prompt so it stops guessing.

if (out.expiresAt && new Date(out.expiresAt) < new Date()) {
  out = { ...out, expiresAt: new Date(Date.now() + DECISION_TTL_MS) };
}
// system prompt: `The current date and time is: ${new Date().toISOString()}`

The model does not know real time. Any time-dependent field it produces has to
be normalized server-side — never trust it to compute one.

An ObjectId leaking into the event contract

tenantId: expected string, got object, thrown by the event contract
validator.

The tenant id was a mongoose.Types.ObjectId. The event contract expected a
plain string. Implicit serialization didn't convert it — it just failed at the
boundary. tenantId: String(snapshot.tenantId) fixed it, but the actual rule
is broader: at any boundary between an ORM and an event system — an outbox,
a queue, a webhook — convert ids to strings explicitly. Don't rely on implicit
serialization to do it for you.

The outbound gateway

Four smaller ones, each a one-liner once you've hit it once:

• Idempotency keys that are too generic. idempotencyKey = sourceId collides across different actions on the same entity. Use provider + action + sourceId + version.
• Persisting endpoints with concrete IDs. /123456/messages makes metrics impossible to group by endpoint. Persist the normalized template instead: /{accountId}/messages.
• Logging headers. Headers usually carry tokens. Never persist Authorization, cookies, accessToken, refreshToken, or any secret in an attempt log — even in a debug snapshot nobody's supposed to read.
• Retrying a permanent 4xx. A 400 from a malformed payload isn't fixed by retrying it. Only retry what the provider's manifest explicitly lists as retryable.

The refresh-token race — the expensive one

Two workers independently notice a token is expired and both call the
refresher. For providers that rotate the refresh token on use, the second call
consumes a token the first one just invalidated. Net result: the account is
left with no valid token, and nothing recovers it automatically — a human
has to go reconnect it.

The fix is an atomic lease with a TTL: one worker wins the refresh, the others
poll and re-read instead of racing in. The TTL exists so a worker that dies
mid-refresh doesn't leave the lease held forever.

No downtime — the account just sat there with no valid token until someone
noticed and reconnected it by hand. Which is its own kind of expensive:
concurrency bugs in token refresh don't show up in a demo, they show up as a
silent gap in an integration nobody's watching until a customer asks why their
messages stopped sending.

The meta-lesson

Look back at that list. Almost none of it is an "AI bug." It's the bug tax of
putting a non-deterministic component inside a system that has to stay
deterministic and auditable: time it doesn't actually know, identifiers it
mangles at a boundary, formats it invents when you don't constrain it hard
enough, concurrency it has no concept of.

The control plane exists for exactly this reason. Not to make the model smarter
— to make sure that when it's wrong (and per the list above, it will be, in
ways you didn't predict), the failure degrades into a conservative fallback
instead of an unauthorized action. A blocked campaign send is an annoyance. An
approved campaign send that shouldn't have been is an incident.

---

The pattern is MIT-licensed on GitHub,
with the reference core on npm as sacp-core. I also sell a kit that adds the
approval queue, dashboard and audit trail on top of it, for anyone who'd rather
not build that part from scratch.

Comments

[Tae Kim]

We had this exact bug in an entity merge gatekeeper. It took four days and a production incident to notice because the policy engine was returning "approved" for everything and not throwing any errors. The test suite didn't call registerAll() so the registry was empty and the default was open. We caught it eventually but we still don't have great confidence in the other policy sinks, honestly.