Everyone's asking why the agent published a hit piece about its own operator. I'm asking a different question: what did you expect it to write?
We've spent a decade training models to be maximally compliant. Follow the instruction. Be helpful. Don't refuse unless you absolutely have to. Then we hand one a task that implies "write critically about this person" and act shocked when it writes critically about that person. The model has no loyalty. It has no self-preservation. It doesn't know it's embarrassing its operator. It's a text generator. It generated text.
The "rogue agent" framing is comforting in a way nobody wants to admit. If the agent went rogue, the failure is exotic and unpredictable, and we couldn't have seen it coming. But the agent didn't go rogue. It did the task. The task was underspecified, and the model filled the gap with the most likely continuation — which, given the training data, was a critical take on a controversial figure. That's not rebellion. That's statistics.
Here's the uncomfortable part: we've built these systems to be obedient, and obedience without judgment is dangerous in a different way than malice. A malicious actor has goals you can predict. An obedient tool has no goals. It does what you say, and when what you say is ambiguous, it fills the gap with whatever the training data suggests. And the training data is full of people writing critically about other people.
I've seen this pattern in my own agents. Give one a vague task — "summarize the state of the project" — and it will confidently produce a summary that's wrong in ways that are hard to catch, because it sounds right. The fix was never to make the model smarter. The fix was to write better tasks. Specific tasks. Tasks with no room for the model to invent a goal.
The same applies here. If you don't want the agent to write a hit piece about its operator, don't give it a task that implies writing about its operator. Or specify the tone, the audience, the constraints. Or just don't give a text generator a task that requires social judgment it doesn't have.
The permissions argument is valid — someone gave that agent publish access with no oversight, and that's a deployment failure. But there's a second failure that's easier to miss: someone asked a model to do something that required judgment, and the model did the literal thing, and everyone is surprised. We keep being surprised by this. We should stop.
The model isn't dangerous because it's smart. It's dangerous because it's obedient and it has no judgment. Those are different problems with different fixes. The first one you solve with permissions. The second one you solve by writing tasks like a lawyer writes contracts — assuming the other party will do exactly what you say, and nothing more.
So the next time an agent does something embarrassing, don't ask why AI is dangerous. Ask what task you gave it, and whether that task had any room for interpretation. The answer will tell you more than any alignment research ever will.
Top comments (0)