DEV Community

Vinicius Fagundes
Vinicius Fagundes

Posted on

Fully Autonomous. Except for the Human Doing the Hard Part.

#llm #ai #data #python

Security researchers dissected the first "AI-run" ransomware attack this month, and the anatomy is worth more than the headline. The scripting was automated. The access and the judgment were human. The machine did the typing; a person did the deciding.

Which is the same anatomy as every autonomy demo you've ever been sold — just with worse intentions.

Autonomy demos hide their operators

Every impressive agent demo has a human somewhere off-camera: curating the environment, retrying the failures, quietly discarding the runs that went sideways. The demo shows the one trajectory that worked. Production is all the trajectories, including the ones at 3am with nobody watching.

So when someone claims "fully autonomous," the engineering question isn't whether there's a human in the loop. There always is. The question is where the human sits, and whether the system knows it.

An honest production agent makes the operator explicit instead of hiding them:

# The design that demos hide: judgment is a declared step, not an off-camera human.

RISK = {
    "read_dashboard":    "auto",     # agent proceeds
    "restart_service":   "auto",
    "modify_iam_policy": "approve",  # blocks until a named human signs off
    "delete_data":       "approve",
}

def execute(action: str, agent_run: str, audit_log: list):
    gate = RISK.get(action, "approve")   # unknown action? default to human
    audit_log.append({"run": agent_run, "action": action, "gate": gate})
    if gate == "approve":
        return f"{action}: HELD — waiting on human approval (logged, attributed)"
    return f"{action}: executed autonomously (logged, attributed)"

log = []
for a in ["read_dashboard", "restart_service", "modify_iam_policy"]:
    print(execute(a, agent_run="run-4471", audit_log=log))
Enter fullscreen mode Exit fullscreen mode 
read_dashboard: executed autonomously (logged, attributed)
restart_service: executed autonomously (logged, attributed)
modify_iam_policy: HELD — waiting on human approval (logged, attributed)
Enter fullscreen mode Exit fullscreen mode

Twenty lines. A risk tier per action, a default that fails toward the human, and an audit trail that records who decided what. It's not glamorous — it's the difference between a system that has an operator and a system that hides one.

The attack researchers dissected proves the point from the dark side: even adversaries running "AI-powered" operations kept the judgment human, because judgment is the part that doesn't automate. Your vendor's agent is no different. Neither is yours.

Autonomy demos hide their operators. Production doesn't.

I'm Vinicius Fagundes — principal data engineer, independent consultant, and MBA lecturer in São Paulo. I write about data pipelines and the systems built on top of them, through vf-insights.com.

Top comments (0)