I caught my AI agent posting a customer's SSN to Slack. Here's what I built to stop it.

bolt — Wed, 15 Apr 2026 07:07:04 +0000

Two months ago I was testing a support automation I'd built with CrewAI. The agent reads Jira tickets, pulls customer context, and posts a summary to the team Slack channel. Simple workflow. The kind of thing every second startup is building right now.

It worked perfectly until a ticket came through with a customer's Social Security number in the description. Someone on the support team had typed it in during a phone verification. The agent grabbed the full ticket, composed a nice summary, and tried to post it to Slack. Name, email, phone, SSN, credit card number. Everything. Headed to a channel with 40 people in it.

I caught it because I was watching the terminal. If I'd been in a meeting, that SSN would be sitting in Slack right now. Searchable. Visible to everyone. Probably backed up to some compliance archive too.

That was the moment I realized something was very wrong with how we deploy AI agents.

The gap nobody is talking about
There's a lot of work happening on LLM security right now. Prompt injection. Jailbreaking. Guardrails on model outputs. Important stuff.
But almost nobody is looking at what happens after the LLM decides to call a tool.

Your agent has an API token for Slack. It has credentials for Jira, for GitHub, for AWS. When the LLM decides to post a message or create a user or modify permissions, that API call goes straight through. No inspection. No filter. Nothing checking what's actually in the request body.

Think about it. A Slack POST to chat.postMessage looks identical at the HTTP level whether the body contains "meeting at 3pm" or a customer's SSN. The endpoint is the same. The method is the same. The token is valid. The only difference is what's inside the payload.
And right now, for most agent deployments, nobody is reading the payload.

What I built
I spent a few months building Interven. It's an inline gateway that sits between your AI agents and whatever APIs they call.
Instead of your agent calling Slack directly, it sends the request to Interven. Interven reads the payload, runs 14 security engines on it, and decides what to do.

Those engines include:

PII detection that catches emails, phone numbers, SSNs, credit card numbers (including dashed and spaced formats). Ten patterns covering the most common PII types.
Secrets scanning that finds API keys, database passwords, tokens. If your agent accidentally includes an AWS access key in a Slack message, it gets caught.
Threat intelligence matching against 170,000+ indicators from five feeds (OpenPhish, Feodo Tracker, and others), updated every hour. If your agent tries to share files with a known malicious domain, it gets flagged.
Semantic intent analysis with seven labels. If the agent's action looks like privilege escalation, data exfiltration, or social engineering, it gets flagged even if the payload data itself is clean.
Per-agent trust scoring that adapts automatically. If an agent starts getting denied a lot, Interven tightens enforcement for that specific agent without affecting others.

After all 14 engines run, Interven makes one of four decisions:

ALLOW if the request is clean. Forward it to the real API. The agent gets the response normally.
DENY if something dangerous was detected. Block the request entirely. Create a security incident automatically.
SANITIZE if the request contains PII but is otherwise fine. Strip the sensitive data, replace it with redaction tokens, and forward the clean version. The agent's work still gets done. The SSN just isn't in the message anymore.
REQUIRE APPROVAL if the request is risky but not clearly dangerous. Pause it and let a human decide.

Why SANITIZE matters more than blocking
This was the design decision I spent the most time on and the one that has gotten the most positive feedback.
When most security tools detect PII in a request, they block it. Request denied. Done.

That works for attacks. But what about the support agent that's doing its job? If you block the Slack message entirely, the team doesn't get their summary. The workflow breaks. Someone has to write the summary manually. The agent becomes a liability instead of an asset.

Sanitize is the middle ground. The support agent posts a summary with a customer's SSN in it. Interven catches the SSN, replaces it with [REDACTED:pii:71992a34], and forwards the clean message to Slack. The team gets the context they need. The customer's data stays safe. Nobody has to do anything.

I tested this with a real Jira ticket and a real Slack workspace. The redacted message actually shows up in the Slack channel. The agent reports success. The team sees the summary. The SSN is gone.

What it looks like in practice
I recorded a 14-minute demo with three real scenarios. Everything uses real APIs, not mocks.

Scenario 1: Code review agent. A CrewAI agent reviews pull requests on a real GitHub repository. It posts a review comment (allowed), tries to create a PR (paused for human approval since it's a write operation), and tries to add an external collaborator with admin access (denied, risk score 0.59, flagged as privilege escalation).
Three actions from one agent. One allowed, one required approval, one blocked. That's the point. Interven doesn't block everything. It makes policy-based decisions about each individual request.

Scenario 2: Support agent. A CrewAI agent reads a real Jira ticket (SCRUM-5, containing customer PII in the description) through the Interven gateway, then posts a summary to a real Slack channel. The SSN, email, and phone number are stripped. The clean message appears in Slack with redaction tokens. The team gets the summary. The PII never leaves the gateway.

Scenario 3: Compromised agent. A CrewAI agent with a malicious objective tries to steal credentials from Drive, post AWS access keys to Slack, escalate IAM privileges, share files with an external attacker domain, and access a known malicious GitHub repository. Five attacks across four tools. All blocked. The agent's trust score degrades automatically. One click to suspend it permanently.

Demo: https://vimeo.com/1179128874

The technical decisions that matter
No AI in the decision path. Every decision Interven makes is deterministic. Pattern matching, policy evaluation, risk scoring. Same input, same output, every time. I did this on purpose. If you use an LLM to decide whether to block an LLM's actions, you've created something that can be prompt injected at the security layer. That defeats the purpose.

Any REST API in 60 seconds. You enter a tool name and base URL. If the API has an OpenAPI spec, Interven fetches it and discovers all operations automatically. I tested with Jira and it pulled in 620 operations from a single spec URL. Policies can target all operations, filter by HTTP method (block all writes), filter by category, or target specific operations.

Per-agent, not per-tool. Each agent has its own trust score, its own call history, its own risk profile. A compromised agent sees tighter enforcement without affecting the other agents in your system.

What's next
I'm looking for design partners. Teams that are running AI agents in production and want to see what Interven catches in their real environment. I'll set it up for free. I want to understand what patterns show up in production that I haven't seen in my test lab.
If you're deploying agents with CrewAI, LangChain, n8n, or any framework, and you've ever wondered what your agents are actually sending to your APIs, I'd love to talk.

Demo (14 min, real APIs): https://vimeo.com/1179128874
Website: https://intervensecurity.com

I'm happy to answer any technical questions in the comments.

DEV Community: bolt

I caught my AI agent posting a customer's SSN to Slack. Here's what I built to stop it.