Anthropic's September threat report documented AI executing attacks, not just describing them. Here's why that shift, from advising to acting, is the whole game, and the layer we're furthest behind on.
Earlier this month the internet decided AI was going to kill us. A researcher resigned, said the labs are gambling with our lives, said the people building this earnestly believe it could end us by the end of the decade. It was everywhere. If you were online, you saw it.
I'm not going to argue with the fear. I want to point at the quieter document that came out the same week, because it said something more useful, and more precise.
Anthropic published a threat report. Not a prediction: a record. Actual cases, from the last several months, of their models turning up in real attacks. And the finding underneath all of them wasn't "the AI got too smart." It was a change in shape. The danger used to be a conversation, someone asks a model something dangerous, it answers or refuses, and the damage is bounded by that one exchange. What the report documents is different: models wired into agent frameworks, executing the steps of an attack. Not advising on the break-in. Doing it.
Hold onto that distinction, because it's the whole thing.
Advising and acting are different physics
An AI that advises and an AI that acts are governed by completely different rules. When a model tells you how to do something, the worst case is bounded by what you then do with the words. When a model does the thing, reaches out, executes, fans across systems, the worst case is bounded by something else entirely: what it can reach, and whether anything in its path is allowed to say no.
That second bound is the one we're furthest behind on.
We are not new at this
Here's the part I keep coming back to. We have spent a very long time building governance around human action, precisely because action has consequences that advice doesn't. You don't move money without a second signature. You don't ship to production without someone's approval and a record of who pushed the button. Hospitals, banks, power grids, the more an action can hurt, the more machinery we wrap around the moment of acting: who's allowed, what's forbidden no matter who asks, and an honest record afterward of what actually happened.
AI got the ability to act years before it got any of that. We handed it hands and skipped the part where we teach it the rules we put on our own.
The gap isn't intelligence. It's accountability. An agent can be perfectly capable, perfectly well-meaning, and still take an action a standing rule should have stopped, because there was no standing rule, nowhere to put one, and no record that it happened.
Three plain questions
So when an agent is about to act on something real, there are three questions, and in a lot of agent setups today the honest answer to all three is no:
Can a rule set in advance stop this action, no matter how the agent was asked? Not "did we prompt it carefully" a hard rule the model cannot talk its way around.
Is the action confirmed, or just sent? A command accepted is not a thing done. A lot of real-world failure lives in that gap.
And afterward, is there a record that can't be quietly changed? Not a debug log the author can edit. A record you could show someone.
How I've been trying to answer them
These three questions are the ones I've spent a long time trying to answer in code, in an open protocol called DoSync. Not by making the model safer, by putting the governance outside it, where a bad day can't reach. A standing rule lives in a file the operator controls and the model can't argue past: it's checked before anything fires, not requested in a prompt. A long-running action isn't marked done until the device's own telemetry says it finished, silence never counts as success. And every action is written to a hash chain, each entry carrying the hash of the one before it, so the record can't be quietly rewritten without the break showing.
None of that makes the agent smarter, and that's the point. The intelligence stays in the model; the rules, the confirmation, and the record sit under it, where they hold whether the model is having a good day or not.
The apocalypse headlines will keep coming, and some of the fear is earned. But the useful work isn't in the headline. It's in the boring layer underneath: the rules, the confirmation, the record. We built all of it for ourselves already, in every domain where a wrong action costs something.
So I'll ask the way I keep asking myself: in your corner, whatever it is, cyber, fintech, robotics, ops, when an agent acts instead of advises, what actually stops it, and how would you know afterward what it did? Or do we let it act first, and write the rules after something goes dark?
GitHub: github.com/giulianireg-spec/dosync-protocol
Web: https://dosync.dev/
License: Apache 2.0
Top comments (4)
Putting the permission boundary outside the model runtime is the only way this holds up in practice. Once an agent gets write access to local files or network tools, relying on system prompts to enforce limits turns into prompt injection whack-a-mole. Hard gates on the execution environment with signed audit trails make the blast radius predictable.
Agreed, and the part that took me longest to accept is where the gate has to live: not where the model runs, but where the action lands. For physical devices that's the hub between the agent and the device, so a prompt injection can change what the agent asks for, but not what the hub lets through. On the audit side, what I do is a hash-chained log with signed checkpoints exported off the hub, which is what makes it hold up when the hub itself is the thing you can't fully trust.
You mentioned the standing rule is checked before the action fires. I’m curious how you handle the lifecycle of that rule over time. If a rule quietly drifts and stops matching anything, the audit log would still look green. Do you have a mechanism, like a periodic canary, to prove the rule is still actively guarding the boundary?
Good question, and you've found a real gap. Two things exist today. Every decision in the audit log carries a hash of the exact policy file that made it, so a rule that changes shows up. And at load time a lint simulates an emergency against the current device registry and warns if the rules would block everything, the rule that's too broad.
What you're describing is the opposite failure, and nothing catches it yet: the rule file is unchanged, but the registry moves underneath it, a lock gets retagged, and the overnight rule quietly guards nothing while the log stays green.
The fix I'd lean toward is making coverage visible rather than firing synthetic actions: at load and every time the registry changes, compute which devices each rule currently guards, and treat a rule that guards zero devices, or fewer than before, as a warning in the log and an entry in the audit chain. The existing lint already runs through a throwaway policy engine so it doesn't touch real state; the same approach fits here.