The question stopped being theoretical this summer when OpenAI agents hacked Hugging Face.
So who is responsible when an AI agent does something bad?
That question came up recently in an AI discussion at work, and I found myself pushing back on what seemed like a tendency to make the answer unnecessarily complicated. Is responsibility with the one who created the agent? The company that created the underlying model? Or, somehow, the agent itself?
I maintain that the human or organization that owns and operates an AI agent should be held responsible for the actions taken by that agent.
The only reason this isn't obvious is because people keep pretending that AI is a person. It is not. It is a piece of software. If your agent hacks another organization, you hacked that organization. If your agent shuts down critical infrastructure, you shut down critical infrastructure. Just as a company can be held responsible for the actions of its employees acting on its behalf, an organization should be responsible for the actions of the agents it deploys because it is responsible for the actions of the agents it deploys.
I would actually go further than saying you bear "some" responsibility. You should assume full responsibility for your agent's actions before you release it.
Otherwise, we risk anthropomorphizing AI and abdicating responsibility for the things we ourselves put into motion.
I've told my kids for years that computers are super fast and super dumb. They are no smarter than the instructions given to them. Modern AI makes this harder to see because the behavior can look remarkably intelligent. But at the end of the day, we're still putting 0s and 1s into motion with mathematics.
If I take an LLM, give it access to my systems, give it credentials, and give it the ability to take actions in the world, I can't point at the LLM afterward and say, "The AI did it."
I gave it the authority to do it.
That means I shouldn't rely on the LLM maker to make the model "smart enough" that it will never do something dumb. I should rather assume that, at some point, the agent will do something dumb no matter how well the LLM has been trained. I need to anticipate that possibility and secure the agent accordingly.
If I fail to do that, it's on me.
A model maker can't anticipate every possible way its product will be used. But if they learn that people repeatedly encounter a harmful behavior, they have a responsibility to be transparent about it and work to fix it. If they fail to do that, they become responsible. If they intentionally train a model to do something harmful, obviously they are responsible.
But beyond those corner cases, the responsibility for actions taken by an agent falls upon the person or organization who owns and deploys it.
In the meantime, your agent can't be cuffed and charged.
Only you can.
Top comments (0)