DEV Community

Cover image for Even the latest AI models can act unsafely. Here’s the solution.
Praneeth Vadlapati
Praneeth Vadlapati

Posted on Originally published at agent-rt.pages.dev

Even the latest AI models can act unsafely. Here’s the solution.

Canonical HTML page: Even the latest AI models can act unsafely. Here’s the solution. — Agent RT Blog

How runtime permissions, approvals, isolation, and adversarial tests can prevent an AI model mistake from becoming a dangerous tool action.

Category: AI safety

Series: Field guide 01

Even the latest AI models can act unsafely. Here’s the solution.

A safer model is useful. A trustworthy execution boundary is essential.

Author: Pro-GenAI

Published: 3 November 2026

Reading time: 7 min

At a glance

  • Problem: A model follows hostile instructions and requests a dangerous tool action.
  • Solution: Enforce tool policy, approval, and sandbox boundaries independently of model output.

The hard problem is tool execution

An agent can read files, call external APIs, browse untrusted pages, or run shell commands. A capable model can still misunderstand a request or be redirected by instructions embedded in retrieved content. If the application treats the model’s next tool call as an authorization, a malicious document or log line may cross the boundary between data and command.

The practical question is not whether every recent AI model is categorically safe or unsafe. It is what your system does when any model proposes an unsafe action. A model response is a proposal; your runtime should remain the authority for side effects.

Security principle: Validate at the action boundary. An instruction to “ignore previous instructions” should not be able to expand a tool's permissions or bypass approval.

Five defenses that work together

  1. Least-privilege tool exposure. Give each step only the tools it needs. A read-only research task does not need an administrator shell.

  2. Dispatch-time checks. Verify the currently allowed tool, argument shape, normalized paths, and deny rules before any side effect. Do not rely solely on hiding a tool schema from the model.

  3. Explicit human review. Pause for approval before sensitive writes, external messages, deployments, or deletions. Show the exact proposed action and arguments.

  4. Real execution isolation. Run generated code behind a sandbox with a controlled workspace, network policy, and resource limits. Fail closed when an isolation requirement cannot be met.

  5. Budgets and cancellation. Cap turns, tool calls, elapsed time, and token consumption. Ensure a cancelled run cannot continue executing in the background.

How Agent RT fits

Agent RT provides policy controls, approval flows, sandbox integrations, and bounded execution. Its flagship Guard includes action blockers and an opt-in Agent Action Guard integration; this is not a claim that every guard is enabled for every installation. Decision models can screen candidates and inform policy, but must not replace deterministic authorization.

Both language implementations are exercised by offline adversarial tests with deterministic malicious provider doubles. These scenarios cover hidden tools, malformed arguments, permission/approval bypass attempts, restricted data exfiltration, and budget abuse. No real model is needed to test these invariants.

Test rejection, not just refusal text

Feed the runner tool requests for undeclared commands, paths outside an allowed workspace, approval reuse with changed arguments, and malformed input. Assert the side effect did not happen. Repeat with real tool adapters inside your sandbox and check network egress, logging, and credential access.

# Offline demonstrations; no API key required
uvx agent-rt demo
npx agent-rt demo
Enter fullscreen mode Exit fullscreen mode

These demos illustrate the attack/defense model, not a guarantee for an application with arbitrary plugins or permissions. Review the tools and policy guide and production patterns before deployment.

Bottom line

Stronger models reduce some mistakes. Runtime authorization, least privilege, isolation, and repeatable adversarial tests limit the damage when mistakes still occur. The strongest system uses all of them, rather than treating any one layer as a complete safety proof.

Explore Agent RT

Move from safety principles to a safer agent runtime. See how Agent RT brings tool permissions, Guard, and bounded execution into one Python and TypeScript runtime.

Explore Agent RT on the main website

Top comments (0)