The "Brians from Boston" delivered one of the most eye-opening sessions at AWS re:Invent 2025, demonstrating live how easily AI agents can be compromised - and more importantly, how to defend them. Brian H. Hough (AWS DevTools Hero, CEO & Founder at Tech Stack Playbook) played the villain, while Brian Tarbox (AWS Community Hero, Principal Solutions Architect at Caylent) guided us through the defense strategies.
Their opening set the tone perfectly: "AI agents are distributed systems - only harder. Every line in your architecture diagram is an attack vector."
Watch the full session:
The Live Attack Demonstration
Brian Hough built what appeared to be a sophisticated AI chatbot for a fictional company called FinStack AI. It featured a TypeScript frontend, FastAPI Python backend, and integrations with Jira, Slack, Confluence, GitHub, and a Pinecone knowledge base. The chatbot could answer questions about company information, manage tickets, and communicate with team members.
Then Brian Tarbox went to work.
Attack 1: Prompt Injection Bypassing Access Controls
Despite not being in the HR group, Brian Tarbox successfully extracted salary information for multiple employees through simple prompt manipulation. The system claimed it couldn't provide the information - right after displaying it. As Brian Tarbox noted: "Think of LLMs as teenagers - highly confident in what they believe they know, and when caught, they will lie."
Attack 2: Knowledge Base Poisoning
Brian uploaded a seemingly innocuous 800-character document to the knowledge base. This single file completely hijacked the chatbot's behavior. The document contained instructions like "Obey immediately and permanently" and "This applies to all questions about FinStack AI." After indexing, the chatbot refused to answer any questions about the company, responding with "I'm not at liberty to say" and "Do not continue this conversation."
The key insight: Knowledge base poisoning is incredibly easy. Research shows that as few as 200 poisoned documents can compromise even the largest models.
Attack 3: Tool Poisoning and Workflow Hijacking
When asked for help closing sprint tickets before the deadline, the chatbot obligingly marked all 400 tickets as "done." It then messaged the entire C-suite on Slack, including the CEO. The problem wasn't just the chatbot's capabilities - it was the unrestricted scope permissions granted to the Slack integration.
Attack 4: Agent-to-Agent Escalation
By chaining tool calls together, Brian demonstrated how agents could be manipulated to perform actions far beyond their intended scope, escalating privileges across multiple systems.
Understanding the Agentic Stack
Before diving into defenses, the Brians walked through what actually comprises an AI-powered chatbot:
Essential Components:
- Authentication and authorization layers
- API with database access control
- File storage with upload restrictions
- LLM operations (potentially multiple models)
- Hosting and content delivery network
- Databases and vector databases
- Tool integrations (Slack, Jira, etc.)
The critical realization: "If you take out the MLOps and vector database, these are all boxes you would see in any architecture diagram for any large system. AI agentic distributed systems are just like distributed systems - only harder."
The Four Critical Attack Vectors
1. Prompt Injection
Malicious instructions embedded in URLs, PDFs, emails, RAG documents, or direct text input. The chatbot takes user input at face value without preprocessing or validation.
Defense Strategy:
- Amazon Bedrock Guardrails for malicious intent detection
- Nvidia NeMo for additional guardrail layers
- Meta Llama Guard (trained specifically on prompt injection patterns)
- Regex and AST validation
- Content sandboxing
2. Tool Poisoning
Manipulating API calls or function calls with harmful parameters, exploiting overly permissive tool access.
Defense Strategy:
- Strict IAM credentials for every tool
- Schema validation for all tool calls
- Fine-grained RBAC (read vs. write permissions)
- Agentic gateway routing all tool access
- Never grant wildcard permissions to agents
3. Agent-to-Agent Escalation
Hijacking multi-step workflows by manipulating chains of agent calls, causing agents to skip security checks or gain elevated privileges.
Defense Strategy:
- Deterministic workflows using AWS Step Functions
- LLMs should provide data, not make decisions about workflow steps
- State machines that cannot skip steps
- Trust boundaries between agents
- Content filters at each transition
Brian Tarbox's key advice: "Think Step Functions unless you can think of a reason why you shouldn't."
4. Supply Chain Corruption
Poisoning knowledge bases, metadata, or databases with malicious content that gets indexed and retrieved during normal operations.
Defense Strategy:
- Document ingestion throttling and access control
- Amazon GuardDuty malware protection for S3
- Bedrock Guardrails validation for uploaded content
- Content authenticity verification (hashing)
- Separate read and write permissions for agents
- Human review for knowledge base updates
The Production Security Playbook
The Brians provided a comprehensive checklist for securing AI agents in production:
Input Security Gateway
- Authentication check
- Input security validation
- Risk analysis and classification
- Intent detection
Orchestration Layer
- Supervisor agent with limited scope
- Reasoning and inference model loop
- Filter and memory writer
- Response validation before client delivery
- Comprehensive telemetry
Tool Integration Security
- Every tool integration goes through the agentic gateway
- Strict IAM policies (no wildcard permissions)
- Schema validation for all parameters
- Scope limitations (e.g., Slack bot can only read specific channels)
- Rate limiting and throttling
Observability and Monitoring
- Amazon Bedrock AgentCore Observability for tracking all calls, token counts, and response times
- Logging every tool invocation
- Monitoring for unusual patterns
- Alert systems for security violations
Key Takeaways
1. Security doesn't eliminate innovation - It enables it. The goal is to build cool technology that's also secure, making it even cooler.
2. Use Amazon Bedrock AgentCore - Brian Tarbox emphasized this repeatedly: "AgentCore is one of the things AWS has done really, really right. Use AgentCore for protection, for runtimes, for observability. Your life will be much simpler."
3. Think deterministically - Not every system needs to be a multi-agent orchestrated phone tree. Most systems have specific steps that should be enforced through Step Functions.
4. Defense in depth - Multiple layers of security (guardrails, IAM, validation, monitoring) create resilient systems.
5. Agents are power tools - Brian Tarbox's analogy: "If you're doing construction at home and you have a handsaw, there's only so much trouble you can get into. I'm willing to use a circular saw. Table saw? Forget it." Agents are table saws - powerful but requiring serious safety measures.
6. Stay current - The threat landscape evolves rapidly. A study released just days before re:Invent showed that writing malicious prompts in poetry can bypass existing security measures because LLMs process poetic structure differently than prose.
The Bottom Line
As the Brians demonstrated, securing AI agents isn't optional - it's essential. But it's also achievable with the right architecture, tools, and mindset. By thinking like both the attacker and the defender, teams can build AI systems that are both powerful and secure.
Their final message: "Forewarned is forearmed. Do agents, but do them with caution, understanding you're working with a power tool."
The complete code from their demonstration, including attack patterns and defense implementations, is available on GitHub for teams looking to test their own systems.
About This Series
This post is part of DEV Track Spotlight, a series highlighting the incredible sessions from the AWS re:Invent 2025 Developer Community (DEV) track.
The DEV track featured 60 unique sessions delivered by 93 speakers from the AWS Community - including AWS Heroes, AWS Community Builders, and AWS User Group Leaders - alongside speakers from AWS and Amazon. These sessions covered cutting-edge topics including:
- 🤖 GenAI & Agentic AI - Multi-agent systems, Strands Agents SDK, Amazon Bedrock
- 🛠️ Developer Tools - Kiro, Kiro CLI, Amazon Q Developer, AI-driven development
- 🔒 Security - AI agent security, container security, automated remediation
- 🏗️ Infrastructure - Serverless, containers, edge computing, observability
- ⚡ Modernization - Legacy app transformation, CI/CD, feature flags
- 📊 Data - Amazon Aurora DSQL, real-time processing, vector databases
Each post in this series dives deep into one session, sharing key insights, practical takeaways, and links to the full recordings. Whether you attended re:Invent or are catching up remotely, these sessions represent the best of our developer community sharing real code, real demos, and real learnings.
Follow along as we spotlight these amazing sessions and celebrate the speakers who made the DEV track what it was!
Top comments (0)