This one is less technical and I think it matters more than the technical ones.
A user instructs an agent. The agent takes an action on a third-party service. The action turns out to be wrong — the wrong plan bought, the wrong data shared, terms accepted that the person would not have accepted.
Who is accountable? The user who gave a vague instruction? The agent operator whose model interpreted it? The service that accepted the request without checking there was a human behind it?
Right now the honest answer is that nobody has decided, and every party has an incentive to point at the other two.
What makes it more than a philosophical question is that it determines what we should be building. If the service is accountable, we need to be able to prove there was human intent behind a request — which is an engineering problem with real answers emerging. If the operator is accountable, they need auditable records of what was asked and what was done. If the user is accountable, then agents need to be far more explicit about what they are about to do than most currently are.
I lean towards: the operator carries it, because they are the only party who can see both the instruction and the action. But that puts enormous weight on agent operators, and I am not sure they have signed up for it.
Where do you think this lands? And is anyone building as though they already know?
Top comments (0)