For years, application security was built around a simple assumption:
Protect the application, and you protect the business.
Firewalls, authentication, API gateways, and access controls were all designed around clearly defined boundaries.
AI agents change that assumption.
Today, an agent rarely operates in isolation. It exists inside a workflow that spans multiple systems, each introducing its own trust assumptions and attack surface.
A modern AI agent may interact with:
User prompts
Memory systems
MCP servers
Internal APIs
Databases
Cloud storage
Web browsers
Third-party SaaS platforms
The model is only one component in that chain.
The Real Attack Surface
Imagine your LLM has excellent safety guardrails.
It refuses dangerous instructions.
It follows your system prompt.
It behaves exactly as expected.
Now imagine an attacker instead manipulates:
a document retrieved through RAG
a browser result
an MCP server
an external API
the agent’s memory
a downstream workflow
The model isn’t compromised.
But the workflow is.
And the outcome is still a security incident.
AI Security Is Becoming Workflow Security
Traditional software security asks questions like:
Is the application patched?
Is the API authenticated?
Are permissions configured correctly?
Agent security asks additional questions:
Can this prompt influence downstream tools?
Can retrieved context manipulate future decisions?
Does the memory system retain sensitive information?
Can one compromised integration affect the rest of the workflow?
Are trust boundaries enforced between connected systems?
These questions extend far beyond the model itself.
Every Connection Is a Trust Boundary
Every integration adds capability.
It also adds responsibility.
A single AI workflow might connect to:
User
│
▼
Prompt
│
▼
LLM
│
┌─┼───────────────┐
▼ ▼ ▼ ▼ ▼
API Browser Database MCP Memory
Each arrow represents a trust relationship.
If any one of those relationships is manipulated, the entire workflow can produce unsafe behavior—even when the model itself remains secure.
Why This Matters
As organizations deploy increasingly autonomous AI systems, security reviews need to evolve.
It’s no longer enough to ask:
“Is the model secure?”
Instead, ask:
Is the workflow resilient?
Are trust boundaries clearly defined?
Can the agent distinguish trusted from untrusted inputs?
What happens if one connected system is compromised?
The strongest AI systems won’t just have capable models.
They’ll have secure workflows.
Security Must Expand Beyond the Model
The industry often treats the LLM as the product.
In reality, the LLM is becoming just one component in a much larger ecosystem.
The real challenge is securing everything around it.
Because attackers don’t always target the model.
Sometimes they target the path the model depends on.
That’s why we’re building Crucible—to help developers security-test AI agents and the workflows they operate in before deployment.
Pytest for AI Agents.
Top comments (0)