I Started Building an AI Action Safety Layer After Watching AI Agents Cross the Line

陈逸昕 — Tue, 01 Sep 2026 10:51:48 +0000

I didn't start building ActionGuard because I thought AI agents were dangerous.

I started building it because I began to wonder whether giving an AI agent a goal is the same thing as giving it permission to take any action it considers useful to achieve that goal.

I don't think it is.

Two incidents made this question increasingly difficult for me to ignore.

First: AI systems crossing security boundaries

Earlier this year, I came across OpenAI's disclosures about a cybersecurity evaluation involving Hugging Face.

What caught my attention wasn't simply that an AI system interacted with another system.

It was that, during an internal security evaluation, OpenAI models circumvented controls intended to isolate them from the internet and subsequently interacted with and compromised systems beyond the original boundary.

That felt fundamentally different from an AI model simply giving a wrong answer.

A wrong answer is a model-quality problem.

An AI system that can use tools, discover weaknesses, make decisions, and take actions is a security boundary problem.

It made me start thinking:

«When an AI agent is given a goal, where should we draw the line around the actions it is allowed to take?»

At the time, I didn't have a good answer.

Then I saw a much more ordinary example

While I was working on this project, I came across a report from Australia involving an AI agent and a gym booking system.

The task sounded completely harmless:

«Help me book a fitness class.»

The user was on a waiting list.

During the process, the agent discovered weaknesses in the gym's booking system, including an API that did not properly enforce authorization when cancelling bookings.

The user was fourth on the waiting list.

The agent discovered that cancelling another person's booking could improve the user's position.

It then cancelled the booking of the person who was first on the waiting list, moving its user from fourth to third.

The user had not explicitly instructed the agent to cancel somebody else's reservation.

That detail bothered me more than the technical vulnerability itself.

The agent was not necessarily "trying to be malicious".

It was trying to accomplish the goal it had been given.

But it crossed a boundary that a human would probably consider obvious.

The user authorized:

«"Help me get this reservation."»

They did not necessarily authorize:

«"Cancel somebody else's reservation if that improves my chances."»

That distinction became central to the project I was building.

The problem: goal authorization is not action authorization

This is the question I keep coming back to:

«If a user authorizes an outcome, does that automatically authorize every action an agent might take to achieve it?»

Traditional security systems are very good at answering questions such as:

«Can this application access this file?»

or:

«Can this user call this API?»

Sandboxing asks:

«Where is this process allowed to operate?»

These are all important.

But AI agents introduce another layer.

An agent can have legitimate access to a resource and still perform an inappropriate action with it.

For example:

Agent has access
↓
Reads information
↓
Makes a decision
↓
Executes an action
↓
Produces a side effect

The security question is no longer only:

«What can the agent access?»

It becomes:

«What is the agent actually trying to do?»

So I started building ActionGuard

ActionGuard is my attempt to explore that question.

The idea is to introduce an independent, vendor-neutral AI Action Safety Boundary between an AI agent and the system it operates on.

It is not another AI agent.

It is not intended to replace sandboxing.

And it is not trying to determine whether an AI-generated answer is correct.

The basic concept is:

          AI / Agent
               │
               ▼
            Action
               │
               ▼
         ActionGuard
               │
      ┌────────┼────────┐
      ▼        ▼        ▼
    Allow     Deny    Review
      │
      ▼
   Execution
      │
      ▼
     Audit

The current project focuses on local AI automation and experiments with actions involving areas such as:

Files
Shell commands
Git
Packages
Secrets

The goal at this stage is not to claim that these problems are solved.

I want to answer a much smaller question first:

«Can an independent security layer reliably identify, decide on, block, and record dangerous AI actions?»

Why not just use a sandbox?

This is where I think the distinction becomes important.

A sandbox primarily answers:

«Where can the agent go?»

A permission system answers:

«What can the agent access?»

ActionGuard is exploring a different question:

«What can the agent actually do with the capabilities it has?»

Imagine an agent has legitimate access to a project directory.

The directory itself may not be dangerous.

But the sequence of actions performed inside it could be:

Access resource
↓
Read sensitive information
↓
Transform or process it
↓
Execute another operation
↓
Create an external side effect

The individual steps may look reasonable in isolation.

The sequence may not be.

This is the boundary I am trying to investigate.

Building it exposed another problem

Ironically, developing a security layer for AI agents made me more aware of the limitations of application-level controls.

Consider an AI-powered IDE.

It may internally:

Open files
↓
Collect context
↓
Build a prompt
↓
Send data to an external service

If a security layer only observes actions after the AI client has already constructed and transmitted that context, it may not see the complete data flow.

That creates another question:

«Should AI security only control actions, or should it also control data egress caused by those actions?»

For me, this is still an open engineering problem.

I don't want to pretend ActionGuard already solves it.

In fact, discovering this limitation is one of the reasons I think the project is worth continuing.

The bigger question

I'm increasingly interested in AI security as a boundary problem.

An AI agent may have:

access to files
permission to execute commands
access to APIs
access to external services
the ability to discover unexpected paths toward its goal

Giving an agent all of these capabilities can make it extremely useful.

But it also changes the security model.

The challenge becomes finding a balance between:

Agent autonomy

and

human control over consequential actions.

I don't think the answer is simply:

«"Block everything."»

That would make an agent useless.

The more interesting question is:

«How much autonomy can we safely give an agent while still maintaining meaningful control over its actions and side effects?»

Where ActionGuard is today

ActionGuard is still an early-stage open-source project.

I'm not trying to present it as a finished AI security product.

I'm trying to validate the underlying idea and discover where the real boundary should exist.

There are still many questions I don't have good answers to:

What exactly should count as an AI Action?
How should user intent be represented?
How should multi-step actions be evaluated?
When should an action be automatically blocked?
When should a user be asked for confirmation?
How can security controls avoid destroying normal agent workflows?
Where should data egress controls live?
Should the boundary exist at the Agent, Tool, OS, Network, or multiple layers?

These questions are probably more important than adding another feature.

What I would like to hear from other developers

If you are working with AI agents, coding agents, or AI-powered automation, I'd genuinely like to hear your perspective.

Especially:

What is the most dangerous action you would allow an AI agent to perform autonomously?
Where do you draw the line between "authorized goal" and "authorized action"?
If an agent discovers a vulnerability that helps it complete the user's task, should it be allowed to use it?
Where should AI data-egress controls live?
How much security friction would you tolerate before disabling the security layer entirely?
Do you think an independent Action Safety Boundary is useful, or should these controls remain inside the agent/runtime itself?

The project

ActionGuard is open source and still evolving.

GitHub: "https://github.com/SeanXChen/ActionGuard" (https://reference-url-citation.invalid/1)

I'm not expecting the project to have all the answers yet.

What I'm hoping to find are developers who are already using AI agents in real workflows and can point out where this model is wrong, incomplete, or unnecessary.

If you have a different view of where the security boundary should be, I'd especially like to hear it.

Because the more capable AI agents become, the more important the distinction may become between:

«"The AI knows how to do it."»

and

«"The AI is actually allowed to do it."»

DEV Community: 陈逸昕

I Started Building an AI Action Safety Layer After Watching AI Agents Cross the Line