If you run any kind of scheduled automation against a declared config — "this step runs unattended," "this channel is on autopilot," "this job is automatic" — the declaration itself is not evidence. It's a sentence someone wrote, once, that nothing has re-checked since. We found two separate failures in our own daily content-distribution pipeline that only showed up because we finally wrote the thing that checks the claim against what actually happened, instead of trusting the config to describe itself correctly.
Here's the pattern, and the three checks that catch it.
Failure 1: a target that read as automated and posted nothing
Our distribution config has one row per platform, each with a mode: auto (an adapter posts it unattended) or draft (a human reviews it). One platform was declared mode: auto pointing at an adapter script. For two weeks, that looked correct — the config said automated, so nobody checked further.
It had posted zero of eight drafts. The adapter path it pointed to lived outside the repo the publisher actually read from, and the publisher only ever accepted two platforms. Every run wrote a fresh draft, the config kept claiming full automation, and the ledger of what actually got published sat at zero the entire time. Nothing errored. Nothing alerted. The config and reality had quietly diverged, and the config was the thing everyone was reading.
The check this needed: don't trust mode: auto as a fact. Compute it — drafts written vs. items actually published, per channel, over a trailing window. A channel that's been "automated" for two weeks with a published count of zero isn't automated. It's silently broken, and the only reason to believe otherwise is that nobody looked.
Failure 2: the instruction said "read the list," then pasted the list anyway
Separately, our own playbook for which platforms to target had a bug that's easy to miss because it looks like redundancy, not a conflict: the instruction said "read the target list from config," and then — a few lines later — named a fixed list of platforms as if for clarity. The fixed list won. The actual config had already dropped one platform (its free API had been retired), but the hardcoded list in the instructions hadn't been updated, so drafts for a dead platform kept getting written for weeks after the real config said not to.
This is a specific, nameable failure mode: a fallback that silently overrides the source of truth it was supposedly just restating. It's not a stale value sitting somewhere obscure — it's sitting right next to the correct instruction, which is exactly why it survived review. Nobody reads "check X" followed by a literal list of X and assumes the list might disagree with X.
The check this needed: if an instruction says to read a value from a config, the instruction should never also restate that value. State it once, in the place that's actually read at runtime. The moment the same fact exists in two places, one of them is going to win silently, and it won't be the one you'd pick.
The actual decision: what gets automated, and why
Once you're checking reality instead of trusting declarations, the harder question is which channels deserve full automation in the first place — and "which adapter was easiest to build" is the wrong answer, even though it's the one that happens by default.
The better split is by what a mistake costs and whether it's reversible:
- Full automation fits channels where credentials are scoped and individually revocable, the format caps posts short enough to force discipline (a hard character limit that refuses rather than truncates is a feature, not a limitation), and a bad post is one tap to delete.
- Human-gated fits channels where a post reads as an identity statement to strangers, a mistake is slow or impossible to walk back, or the audience is small enough that volume itself is a cost — sending something mediocre every single day trains the one human reviewing it to stop reading, which is worse than sending nothing.
And a third category that's easy to misclassify: your own approval queue is not a distribution channel. If the surface where a human approves or rejects drafts gets counted as a "platform" in your coverage metrics, your automation looks more connected than it is — you're counting the review step as if it were a destination.
Packaging it
We eventually pulled this whole pattern — the config contract, the adapters, the daily plays that write drafts, and the verifier that checks claims against the publish ledger instead of trusting the config — into a standalone kit (Distribution Autopilot), because we'd rather hand someone a system that already checks itself than have them discover failure 1 two weeks in, the way we did.
But the two checks above don't need that kit or any kit: compute your automation claims against your actual output ledger, and never let an instruction both reference a source of truth and restate it. Either one, applied to whatever scheduled thing you're running right now, will probably find something.
Top comments (0)