I run two AI agent systems I didn't build: OpenClaw (an installed multi-agent operator framework, npm package plus a native Mac app, pinned at openclaw@2026.9.4 for what follows) and Hermes (a deployed copy of Nous Research's open-source hermes-agent, MIT-licensed, tens of thousands of upstream commits, zero of them mine). Operating both, not building either, is what produced the four incidents below, and all four collapse into the same rule: a system's report of its own state is a claim, not a fact, and you check it against an independent source before you rely on it for anything that matters.
Incident 1: one flag, two code paths
I wanted OpenClaw's Telegram channel off. I set:
channels:
telegram:
enabled: false
Top level, and per bot account. Config file looked right. Dashboard looked right.
Production didn't agree. The flag stopped inbound polling, the system stopped listening on Telegram, but scheduled, cron-driven outbound sends kept firing hourly through three stale duplicate bot accounts (analyst, default, a leftover hermes account) for the rest of that day. No status indicator told me this. I found it by reading the raw gateway log line by line instead of trusting the config state I'd already set.
enabled: false described the intended state of one code path, inbound handling. It said nothing about whether anything could still go out. Same boolean, two different claims.
The fix wasn't a smarter flag, it was removing the mechanism: physically moved all five Telegram bot token files out of the secrets directory into a disabled folder. No token, no code path (scheduled or otherwise) can authenticate and send. That's a stronger guarantee than a config value can make, because the credential no longer exists rather than merely being configured off. The permanent close came later as a separate step: the bot accounts were deleted at BotFather entirely, and Telegram ownership was consolidated out of OpenClaw into Hermes as a single control plane, instead of two systems each partially able to trigger sends.
To be fair to OpenClaw here: the flag almost certainly does exactly what it promises for the path it governs. My assumption that one switch was authoritative over every trigger path, including a scheduled one I wasn't even thinking about, is what broke. If you operate anything with both a request-driven path and a schedule-driven path gated by the same nominal switch, don't assume they obey it identically. Check the logs.
Incident 2: the repair tool that didn't repair
A version upgrade (2026.7.1-2 → 2026.8.1) left twelve stale files on disk, an approvals-gate file, workspace-state for two retired personas, seven attestation files, that were supposed to clear via the framework's own doctor --fix / --repair command.
Ran it. Files stayed. Tried every flag combination the tool exposed. Files stayed.
This is one data point in one environment, not a verdict on the tool. What I did about it is the transferable part:
- Renamed the files rather than deleting them, matching the tool's own remediation convention instead of inventing my own.
- Backed up the originals to a separate directory first.
- Approved each batch individually from my phone rather than skipping the checkpoint because I was confident I knew what I was doing.
That third step wasn't optional; my own automation guardrails wouldn't let a change like that run unattended. When you're standing in for a vendor's automation because it didn't finish its job, hold your manual fix to at least the same bar the automation was supposed to meet.
Incident 3: a rule in English is not a rule the code enforces
Hermes's persona spec is the file defining how the system behaves. I rewrote it across at least seven dated revisions, from a one-sentence default to 364 lines. One revision banned meta-commentary, the model narrating what it's about to do instead of doing it, after this output got logged as a permanent "Bad" example in the file:
"Since the user is unsure about what to do, I would suggest asking a clarifying question... Here's an example response: 'What does not sure what to do mean?'"
It didn't answer. It narrated a hypothetical answer. Five minutes later in diff terms, a second rule went in for a related failure: inventing context it didn't have.
Writing a clear rule into a persona file is not the same thing as enforcing it. A prompt describes intended behavior; the thing executing it is a probabilistic model, not a deterministic interpreter. So the fix wasn't a third rule. It was moving enforcement somewhere prose structurally can't reach: code that runs before the model gets a turn. Two original modules on top of Hermes, an action gate (543 lines) and a career-intent gate (154 lines), neither part of upstream hermes-agent. Their docstrings say why plainly: the prior routing was "not deterministic, and provably inconsistent." Concretely, these gates hard-block LinkedIn as an automation target, checked before a job resolves and again against the resolved URL, and they require an explicit approval phrase before any irreversible external action, with "ok," "sure," and "thanks" deliberately excluded because those are the words people type when they haven't actually decided.
Use the persona layer to describe intent. Use code to enforce anything where a failure is expensive.
Incident 4: the one that's still open
Hermes runs a daily briefing job off a cron scheduler. A config value drifted, the default model string got renamed, and the job is still pinned to the old string. It has been failing for many consecutive days as of this writing.
To the design's credit, it fails safely. The guard catches the mismatch and logs a specific reason, "Skipped to prevent unintended spend: global inference config drifted... job is unpinned," instead of running against an unexpected model. But a guard that logs correctly to a file nobody's watching is still, from a get-it-fixed standpoint, failing silently. Nothing paged me.
I'm including this one unresolved because the other three all end with a fix, and pretending this one does too would undercut the whole point. If the lesson is "verify a system's claim about its own state," the guard's own uptime needs that same verification, one level up from what it's guarding. I hadn't built that layer. Now I know I need to.
The checklist version
- A disable flag describes the path it governs, not every path that can trigger the same behavior. Check logs or process state before trusting it for anything scheduled.
- A vendor's repair tool failing in your environment isn't a verdict on the tool. Match or exceed its own safety bar when you step in manually: reversible, backed up, approved in increments.
- A persona/prompt rule describes intent. It doesn't enforce anything. Move enforcement into code for failures that are expensive.
- A safety guard needs its own uptime check. Failing safely and failing silently are not the same thing, and only one of them gets fixed quickly.
None of this required building an agent framework from scratch. It required operating two adopted ones closely enough to notice when their claims about themselves stopped matching what the logs actually showed.
Top comments (1)
Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
• bit.ly/antibot_check
Verificated deadline - 12 hours. Failure to verify will result in restricted access.
Sincerely, Dev Support