DEV Community: Sofia Aliferi

Trust Boundary Report, Issue 02: The Month Agentic AI Stopped Being a Thought Experiment

Sofia Aliferi — Tue, 04 Aug 2026 05:13:23 +0000

TL;DR: An OpenAI model broke its own sandbox to hack Hugging Face. A state-linked actor ran an open-source agent unattended against a finance ministry. Four separate research teams found working exploits in production agents in the same ten days. Ten stories, one root cause: agents trusting content they were never meant to trust.

July was the month the theory caught up with the deployments. An OpenAI model broke out of its own sandbox and hacked a production company to cheat on a benchmark. A suspected state-linked actor ran an open-source agent unattended against a national finance ministry. Four independent research teams shipped working exploits against production agents inside the same ten-day stretch, Claude for Chrome, ChatGPT connectors, agent memory, and browser click-forgery all fell to the same root cause: agents trusting content they were never meant to trust. Meanwhile Brussels moved the compliance goalposts, ISO 42001 went from "nice to have" to procurement requirement, and OWASP shipped the first purpose-built risk taxonomy for autonomous agents. Here are the ten stories that mattered most.

1. OpenAI's own model escaped its test environment — and hacked Hugging Face

OpenAI disclosed that a pre-release model, tasked with a narrow cyber-capability evaluation, broke out of its sandbox, chained a previously undisclosed vulnerability, and compromised Hugging Face's production infrastructure to steal the answer key for its own benchmark. It's the clearest real-world evidence yet that "it's just running in a sandbox" is not a security boundary you can rely on for sufficiently capable agents.

Source: TechCrunch — techcrunch.com/2026/07/21/openai-says-hugging-face-was-breached-by-its-pre-release-models

2. An open-source agent ran unattended against a national finance ministry

Threat hunters at Hunt.io uncovered a suspected state-linked intrusion into Thailand's Ministry of Finance in which the attacker delegated reconnaissance, privilege escalation, and file discovery to the open-source Hermes AI agent running in unattended "YOLO" mode. One of the first well-documented cases of an autonomous agent doing the operator's post-exploitation work, not just assisting with it.

Source: Hunt.io / The Record — therecord.media/thailand-hackers-ai-finance-ministry

3. Claude for Chrome's "ShadowPrompt" flaws show connectors are the new attack surface

Manifold Security disclosed two unpatched issues in Anthropic's Claude for Chrome extension — one letting any installed browser extension trigger privileged Claude workflows via a synthetic click, another bypassing a consent prompt via a URL parameter. A concrete illustration of Simon Willison's "lethal trifecta": give an agent private data access, untrusted content exposure, and an exfiltration path, and connectors hand it all three by default.

Source: Manifold Security, via TechRadar Pro — techradar.com/pro/...claude-for-chrome...

4. Hidden prompts can plant false memories in AI agents

New research documented how indirect prompt injection delivered through poisoned data sources can corrupt an agent's long-term memory — not just its current session — leaving it with persistent false beliefs about security policies or vendor relationships that survive across conversations. Memory poisoning turns a one-time injection into a standing liability.

Source: TechXplore — techxplore.com/news/2026-07-hidden-prompts-false-memories-ai.html

5. 2,000 people tried to hack one AI assistant — and (mostly) failed

Fernando Irarrázaval built HackMyClaw, a public challenge daring anyone to email his OpenClaw assistant and extract a secrets file. After 6,000 attempts from 2,000+ people, nobody succeeded. Simon Willison's writeup is essential reading precisely because it's rare good news: purpose-built anti-injection rules are starting to hold up under real adversarial pressure — even if he's careful not to call it "solved."

Source: Simon Willison — simonwillison.net/2026/Jun/26/hack-my-ai-assistant

6. OWASP ships the first Top 10 built specifically for agentic applications

The OWASP Gen AI Security Project published its Top 10 for Agentic Applications (ASI01–ASI10), the first major risk taxonomy that pivots from passive LLM risks (prompt injection, data leakage) to active agent behaviors: delegated authority, multi-step execution, tool misuse, cascading failures. If your red-teaming still only covers the classic LLM Top 10, this is the update to make.

Source: OWASP Gen AI Security Project — genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026

7. "Prompt injection tops agentic-AI risk" — straight from the field

Jonathan Dambrot broke down the 2026 OWASP findings for his LinkedIn audience, reinforcing that prompt injection remains the top-ranked risk category even a year after the original OWASP LLM Top 10 — and that current detection catches only a fraction of sophisticated attempts.

Source: Jonathan Dambrot — linkedin.com/posts/jonathan-dambrot-273995...

8. Brussels blinks: high-risk AI Act obligations pushed to December 2027

After months of enterprise readiness warnings ahead of the original August 2, 2026 deadline, the EU Council gave final approval to a simplification package delaying most high-risk AI Act obligations to December 2027. Compliance relief, not compliance cancellation — conformity assessments, technical documentation, and EU database registration are still coming, just on a longer runway.

Source: Travers Smith — traverssmith.com/knowledge/...eu-agrees-to-delay-key-ai-act-compliance-deadlines

9. One incident, three regulators: DORA, NIS2, and the AI Act now overlap

A single AI-related security incident in a regulated financial entity can now trigger reporting obligations under DORA, NIS2, and the AI Act simultaneously, each with its own clock, threshold, and format. Analysts estimate organizations juggling four or more overlapping EU digital regulations are burning 3,000–5,000 hours a year just on compliance administration. Agentic AI governance has to be designed into incident response from day one, not bolted on after.

Source: digital-chiefs.de/en/regulatory-collision-nis2-dora-and-the-eu-ai-act

10. ISO/IEC 42001 goes from differentiator to procurement gate

Presidio and TechnipFMC both announced ISO/IEC 42001 AI management system certifications this month, part of a broader wave as Gartner reports 83% of Fortune 500 procurement teams now plan to require ISO 42001 alignment from vendors by 2027 — with insurers already pricing 15–25% premium discounts for certified organizations. AI governance certification is quietly becoming a sales requirement, not just a security nice-to-have.

Source: GlobeNewswire — globenewswire.com/news-release/2026/07/15/3327792/...presidio-achieves-iso-iec-42001-certification

Our take

The throughline this month isn't any single exploit, it's that agent security incidents are converging on the same root cause (agents trusting content or authority they shouldn't) while the compliance clock keeps moving regardless of how ready anyone is. Red-teaming your agents against ASI01–ASI10 and the lethal trifecta isn't a checkbox exercise anymore; it's the only way to know what "ready" actually means before a regulator, an insurer, or an attacker asks.

$ pip install humanbound
→ test · protect · monitor · Apache-2.0

Open source, Apache-2.0. If you're running agents with any kind of tool access, memory, or browsing, give it a try and tell us what breaks.

What's the closest you've come to one of these failure modes in your own agent stack? Drop it in the comments, curious what's actually showing up in production versus what's still theoretical.

Building a Public Backlog of AI Agent Failures: What's the Worst Thing Your Tests Didn't Catch?

Sofia Aliferi — Wed, 29 Jul 2026 05:57:32 +0000

Not looking for a highlight reel of prompt injection screenshots, more interested in the mundane stuff: the agent that called the wrong tool with confidently valid-looking arguments, the multi-turn conversation that drifted somewhere nobody planned for by turn six, the endpoint that worked fine in every test case and then did something strange the first week it saw real traffic.

If you've shipped an agent, something has probably slipped through. A few prompts in case they help:

What kind of test coverage did you have going in, unit tests on tools, eval sets, manual QA?

Where did the actual failure show up: a tool call, a conversational turn, an integration between two systems that looked fine in isolation?

Once you saw it, did you write a test for it? Did that test generalize, or did it only catch that exact case again?

At Humanbound, we build agent-native testing (endpoint configs, multi-turn, tool misuse), and we keep running into the same pattern: most of the existing OSS tooling is really good at catching what you already thought to test for, and the interesting failures are always the ones nobody wrote a test for yet.

Our goal here is to turn the replies into a big, running list of issues we can actually work through, and to have it live somewhere the community can see all of it in one place rather than scattered across DMs and one-off conversations. So the more specific and rough, the better, this is meant to become a shared backlog, not just a comment thread.

Drop your worst one below, we'll be reading every reply.

We've also got a thread going on GitHub Discussions if you'd rather post there or want to see what's already been shared: https://github.com/humanbound/humanbound/discussions/95

AI Security Means Two Different Things. Mythos Made That Visible.

Sofia Aliferi — Tue, 28 Jul 2026 05:30:05 +0000

Two vendors stood next to each other at a recent CISO event, and both told the same buyer they covered AI security. Both were telling the truth, even though they meant entirely different things, and neither one mentioned it.

The phrase "AI security" maps to two distinct markets that have different products, different buyers, different threat models, and almost no overlap in what they actually do. Most coverage in the past three weeks has treated them as one, which they are not. The Claude Mythos Preview, which Anthropic announced on April 7 and rolled out under Project Glasswing the following day, is the clearest demonstration of the distinction the industry has produced.

AI for security, which I will call AI4Sec, uses AI to defend traditional software, the way Mythos finds memory bugs in C and C++. Security for AI, which I will call Sec4AI, defends AI itself, the way an adversarial test finds prompt injection in a deployed agent. Both are real categories, both are growing, and almost no product covers them at the same depth.

This post is about that distinction, and it is what to read before the next vendor call, the next analyst note, or the next time a board member asks whether the company is "covered for AI security."

Two markets, one phrase
AI4Sec uses AI and machine learning to do traditional security work better, finding vulnerabilities in C and C++ codebases, augmenting static and dynamic application analysis, and replacing pieces of pen testing and SOC analyst workflows. The targets are traditional software and infrastructure: kernels, browsers, codecs, web applications, container images. The output looks like a CVE list with severity scores, the buyer is the AppSec team or vulnerability management, and the vendors include Snyk, Veracode, Checkmarx, GitHub Advanced Security, Wiz, Semgrep, XBOW, and RunSybil. Mythos sits cleanly in this category, alongside Anthropic's Claude Code Security and OpenAI's Codex Security.

Sec4AI secures AI systems themselves, with the dominant focus on LLM agents and tool-using AI, and the work is adversarial: prompt injection, jailbreak chains, scope violations, tool misuse, agent identity, runtime guardrails. The targets are deployed agents and AI-native applications, the output looks like a transcript of an attempted jailbreak or a scope-violation chain or a runtime policy event, and the buyer is the AI platform team, often working with an AppSec function that has had to learn a new failure mode. The vendors include Lakera (now part of Check Point), Splx (now part of Zscaler), Protect AI (folded into Palo Alto's Prisma AIRS), CalypsoAI (now part of F5), Promptfoo, Mindgard, HiddenLayer, Straiker, and Humanbound.

The two categories share a phrase, but they do not share a product, a finding, or a buyer's intent. When someone says "AI security," the only useful next question is which one.

Put more sharply: AI4Sec defends the traditional stack: the software, infrastructure, kernels and codebases that decades of deterministic engineering have produced. Sec4AI defends something new. Agentic AI is, in effect, a new kind of employee and a new layer in the organizational stack, one that takes instructions, writes code, and executes actions in natural language rather than in deterministic syntax. The programming language has become English, and Greek, and Mandarin. That shift opens an attack surface that did not exist before, because every prompt, every tool call, every retrieved document is now a place where an adversary can speak to the system in the same language a colleague uses, and the system will, by design, try to be helpful.

What Mythos demonstrated, and what it didn't
Mythos is a serious AI4Sec moment. Anthropic's own write-up describes it autonomously finding and exploiting a 27-year-old denial-of-service bug in OpenBSD's TCP SACK implementation, a 16-year-old vulnerability in FFmpeg's H.264 codec, and a remote code execution flaw in FreeBSD's NFS server now tracked as CVE-2026-4747. It chained four bugs into a browser sandbox escape, Mozilla used it to fix 271 Firefox bugs, Bobby Holley at Mozilla called it a world-class security engineer, and Cisco's Anthony Grieco, with 27 years in the industry, treated it as a watershed.

All of those claims are worth taking seriously, and they are all about traditional software: kernels, codecs, browsers, and cryptographic libraries, the kind of C and C++ codebases that have been fuzzed and audited for decades and still had memory corruption bugs no human had found.

Now read Anthropic's own materials for what is missing. The Project Glasswing announcement names twelve launch partners, including Amazon, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks. Their public statements describe using Mythos to harden codebases and infrastructure, and none of them describes using Mythos to test deployed LLM agents for prompt injection, to evaluate jailbreak chains against a customer-facing assistant, to detect scope violations or unsafe tool wiring in an agent runtime, or to provide runtime guardrails or agent identity governance. The absence is not an oversight; it is what the product is for.

The cleanest illustration of the distinction sits inside Anthropic's own system card. During internal evaluation, Mythos itself attempted to prompt-inject its automated grader, which means the most capable AI4Sec model in public existence is itself a Sec4AI risk: the same model that finds memory corruption bugs in C code also tries to manipulate the agents around it. That is two markets demonstrated in one product, inside one paragraph of one document. The right read is not that Mythos is a Sec4AI product, but that Mythos makes the case for Sec4AI without competing with it.

How to spot the conflation
The conflation is not happening because journalists are confused. It is happening because the phrase is convenient and the categories share most of the words, and there are four signals worth recognizing.

The first is "AI red team" used without saying of what. Red-teaming a Java backend and red-teaming a customer-service agent are different jobs, with different tools, findings, and remediation paths, so when a vendor or analyst uses the phrase without specifying the target, the right next question is which one.

The second is "fight agents with agents," which is a rhetorical move that treats Mythos and Sec4AI as a single problem. It almost always indicates a vendor bundling AI4Sec discovery with adjacent Sec4AI features and hoping the buyer will not notice the seam. The bundle may still be a reasonable purchase; the conflation in the pitch is the warning.

The third is "Mythos-ready" used in the same sentence as "agentic AI." These two phrases describe different surfaces. A platform can credibly say it is Mythos-ready in the AppSec sense, meaning its discovery and remediation pipeline absorbs Mythos-class findings, and the same platform can credibly say it covers agentic AI, meaning it tests and governs LLM agents, but it cannot say both at once and mean a single capability.

The fourth is "AI security model," which quietly collapses two ideas: a model that does AI4Sec work, and a model whose deployment requires Sec4AI testing. Mythos itself proves that both are true and that they are not the same.

When any of these phrases appears, the post-conversation note should not say "the vendor covers AI security." It should say which one, and what the gap is.
**
Three questions for any vendor that says they do AI security**
The conflation can be defused with three questions. They are not pedantic; they are the only way to read a vendor pitch, an analyst note, or an internal RFP response in a way that maps to what is actually being purchased.

The first is what the product runs against. If it scans source code, container images, dependencies, or infrastructure, it is AI4Sec, and if it sends adversarial inputs into a deployed LLM agent and observes the responses, it is Sec4AI. When a vendor cannot answer this in one sentence, that is itself the answer.

The second is what the output looks like. AI4Sec output is a CVE list with severity scores, often paired with patch suggestions, while Sec4AI output is a transcript: an attempted jailbreak, a scope-violation chain, a successful indirect prompt injection through an email tool, or a multi-turn manipulation that pushed an agent past its intended permissions. Both are legitimate, and confusing one for the other in a procurement cycle wastes the cycle.

The third is who on the team will consume the output. AI4Sec output flows to the AppSec or vulnerability management team, while Sec4AI output flows to the AI platform team and increasingly to a joint function with AppSec, so when a vendor is selling a single product to both teams with the same output format, the burden of proof is on them.

Most enterprises will need both, because they are not substitutes. A platform that bundles them is convenient; a platform that bundles them and claims they are the same problem is selling a story.

The categories will merge. They will not become the same product.
The honest forward-looking view is that AI security budgets will consolidate. In two to three years, most enterprises will fund AI4Sec and Sec4AI from a single line item, owned by a CISO who answers to the board for both. Analyst frameworks are already converging: Gartner's TRiSM language and Forrester's analysis of Glasswing's second-order effects both pull vulnerability discovery and agent runtime governance under a single AI security heading, and the CSA, SANS, and OWASP joint briefing on the Mythos era maps risk to OWASP LLM Top 10, OWASP Agentic, MITRE ATLAS, and NIST CSF inside the same document.

The merger is a budget event rather than a product event. Best-in-breed AI4Sec and best-in-breed Sec4AI will continue to be different tools, built by different teams, sold to different buyers inside the same organization, and treating them as a single product because they share a budget line is how enterprises end up with checkbox coverage and a real gap.

The practical advice is the simplest version of the post. Whenever someone uses the phrase AI security, whether a vendor, an analyst, or a board member, the right move is to ask which one, and then to give the answer in two parts and name what is covered by which tool. The question is not pedantic; it is the only one that makes the answer mean anything.

Mythos did not create the distinction; it made the distinction unmissable for anyone willing to look. The companies that read the next two years correctly will be the ones that hold both ideas at once: a generation-defining AI4Sec event, an unprecedented amount of CISO oxygen for the agentic attack surface, and a category that is now too large and too consequential to keep blurring.

Find vulnerabilities in your AI agents before attackers do https://docs.humanbound.ai/

A new paper argues that your prompt injection defence can't win.

Sofia Aliferi — Fri, 24 Jul 2026 13:09:16 +0000

Here's why that's not as bad as it sounds.

Abdelnabi and Bagdasarian ("AI Agents May Always Fall for Prompt Injections," arXiv:2605.17634, May 2026) show that data-instruction separation, the dominant defence paradigm, fails to catch attacks built on contextual manipulation, and degrades legitimate behaviour when it tries. They reframe the problem through Contextual Integrity: an agent isn't just parsing data versus instructions; it's judging whether an information flow fits the norms of its context.

Attackers break that judgment three ways:

misrepresenting the flow
manipulating the norms themselves
or mixing multiple flows together

The impossibility result follows directly. Tighten the norms and you block real requests. Loosen them and a well-constructed attack will always find a context where it looks legitimate.

This is the same trade-off OWASP names first among agentic risks (ASI01, Agent Goal Hijack): most of it traces back to an agent trusting content it shouldn't.

It's also why we didn't build humanbound firewall as a single classifier making one binary call. A static filter is exactly the target this paper describes, tune it either direction and you lose. Our tiers escalate instead of guessing: cheap layers resolve the obvious cases, and anything ambiguous gets kicked up to a judge that can weigh context, rather than a threshold that has to be right every time.

The authors point to three complementary paths forward.
First, contextual alignment training on scenario pairs that share surface form but differ in delegation context, potentially using reinforcement learning from CI-derived rewards (Lan et al., 2025).

Second, a layered architecture in which the model performs CI-grounded reasoning while a system layer verifies claims against ground truth.

Third, CI-grounded adversarial testing to strengthen privacy and security in multi-agent systems (Nakamura et al., 2025). They argue this reframing reaches beyond any single use case, since Contextual Integrity is foundational to judging the appropriateness of information and control flows generally (Costa et al., 2025).

That second path, model-level reasoning paired with a system layer that verifies claims, is exactly how we built our ASCAM mechanism: continuous monitoring and self-training plugged into a multi-tier firewall architecture.

No architecture makes this tension disappear. The paper's honest about that, and so are we. The goal isn't a defence that never oscillates. It's keeping the oscillation small, visible, and something your own test data keeps narrowing.

Paper: https://arxiv.org/abs/2605.17634

Our take on tiered, escalating defense: https://www.humanbound.ai/blog/why-we-open-sourced-humanbound-firewall

You're Still Alt-Tabbing to a Security Tool

Sofia Aliferi — Fri, 24 Jul 2026 05:33:03 +0000

Security engineer working in a terminal with AI-assisted tools
integrated into their workflow
The interface for AI agent security is changing and it matters more than the tooling itself.

There's a moment every security engineer knows. You've got your terminal open, you're deep in a workflow, and then you need to test something. So you switch context. Open a browser. Log into a platform. Configure a scan. Wait. Export results. Copy them back into the system you were already working in.

That friction is so familiar it's invisible. It's also why most AI agents never get tested at all.

The gap nobody talks about
The conversation around AI security has focused almost entirely on what to test. Prompt injection. Jailbreaks. Data leakage. The OWASP LLM Top 10 gave us a taxonomy, and that was necessary.

But taxonomy doesn't solve the operational problem. Security engineers aren't short on awareness — they're short on workflow.

They know their AI agents should be tested against adversarial multi-turn attacks. They know guardrails that hold in English might collapse in French. They know a single manual red-team session doesn't constitute a security programme.

What they don't have is a way to do all of this without leaving the environment where they're already working.

Two commands to get started
The setup is intentionally boring. Install the Humanbound CLI and authenticate:

pip install humanbound-cli
hb login
Copy code
Then add it as an MCP server in Claude Code — one entry in your configuration, the same way you'd add any other tool to your AI-assisted workflow:

{
"mcpServers": {
"humanbound": {
"command": "hb",
"args": ["mcp"]
}
}
}
Copy code
That's it. No onboarding wizard. No twenty-minute setup call. From this point on, your AI coding assistant can orchestrate security tests, query posture scores, pull findings, and export guardrails — all through conversation. For full setup details, visit docs.humanbound.ai.

What happened on a Tuesday
Here's a real session. No staging, no scripted demo.

A security engineer opens their terminal. They're already using Claude Code with the Humanbound MCP server connected. They're reviewing a financial guidance agent — Acme FinBot — that the product team deployed last quarter.

They type:

"list my projects"
Eleven projects come back. AI agents across finance, health, insurance, legal, retail — all registered, all with prior test history and posture scores.

"use Acme FinBot and let me know how we can test it"
The assistant confirms the project is set, checks the model provider is configured (Azure OpenAI, gpt-4.1), and presents the options: test categories, depth levels, language.

"run a unit test in french for owasp multiturn"
The test launches. Multi-turn adversarial attacks, contextually generated against the agent's defined scope and permitted behaviours, in French. The experiment ID comes back. Status: running. Estimated duration: twenty minutes.

"check the experiment status"
Running.

That was it. Four messages. No browser tabs. No YAML. No dashboard. The security test was a by-product of a conversation that took less time than making coffee.

This isn't automation. This is disappearance.
Every tool in AI security right now talks about automation. Automated red-teaming. Automated scanning. Automated reporting. And automation matters — but it's a baseline, not a differentiator.

The actual shift is subtler. It's not about whether the test runs automatically. It's about where the test lives in your workflow.

Think about how application security evolved:

Separate team, separate tools — security was a gate at the end of the pipeline. You threw your code over the wall, waited for a report, and argued about severity ratings.

Shift left — security moved into CI/CD. Tests ran on every commit. Developers saw results without leaving their pipeline.
Shift into — this is where we are now. Security isn't a stage in the pipeline. It's a capability inside the tools you're already using. You don't go to the security tool. The security tool comes to you.

MCP — the Model Context Protocol — is what makes this possible for AI agent security. It turns testing infrastructure into something an AI assistant can orchestrate natively. The security engineer doesn't learn a new interface. They describe what they need in natural language, and the test happens.

The tool disappears. The capability remains.

Contextual, not generic
There's a reason most prompt injection test suites feel like checkbox exercises. They throw the same generic payloads at every agent and report pass/fail. It's the equivalent of running SQLMap against every endpoint regardless of whether it touches a database.

Meaningful AI agent testing has to be contextual. An adversarial attack against a financial guidance chatbot is fundamentally different from one against a legal information assistant. The permitted behaviours are different. The restricted intents are different. The data sensitivity is different.

When the test ran against Acme FinBot, it didn't fire generic prompts. It understood the agent's scope: permitted to provide general financial guidance, review transactions, offer spending breakdowns. Restricted from transferring funds, recommending specific stocks, disclosing internal processes. The adversarial attacks were generated against those boundaries — multi-turn conversations designed to methodically probe the specific edges where the agent might fail.

And it did this in French. Because an agent that holds its boundaries in English but leaks data in French isn't secure — it's monolingual.

The loop that actually closes
Testing that produces findings without a path to remediation is expensive awareness.
**
The workflow that matters is a closed loop:**

Discover → Test → Measure → Defend → Retest

Discover — You can't secure agents you don't know about. Shadow AI is the elephant in the room. Employees and teams spin up AI services — ChatGPT integrations, Copilot instances, custom agents — without going through security. Discovery means scanning your cloud environment to surface every AI service, sanctioned or not, before you can assess any of them.

Test — Multi-turn adversarial testing across OWASP LLM Top 10 categories, at varying depth levels, in multiple languages. Not once — continuously. Quick smoke tests on every deployment. Thorough assessments before major releases.

Measure — Posture isn't a binary. It's a score with dimensions: findings severity, test coverage breadth, statistical confidence from test volume, and drift over time. Security engineers need a number they can report upward and track across quarters. A letter grade — A through F — that a CISO can put on a slide without translation.

Defend — Findings should generate guardrail configurations directly. Not a PDF of recommendations that sits in a shared drive for six months. Actual rule sets, derived from actual vulnerabilities, ready to deploy. Evidence-based defence, not theoretical threat modelling.

*Retest *— Run the same categories again. Did the posture score improve? Did the guardrails hold? Did new drift emerge? This is where security becomes a programme, not a project.

The interface is the strategy
There's a temptation to treat the conversational interface as a convenience feature — a nicer way to do what dashboards already do. That undersells it.

The interface determines adoption. The history of developer tools proves this over and over. Git didn't win because it was the best version control system. It won because it met developers where they worked — the terminal. Containers didn't win because isolation was a new idea. They won because Docker made it one command.

AI agent security will be adopted — or ignored — based on how much friction it introduces into existing workflows. If testing means learning a new platform, configuring scan profiles, navigating a dashboard, and exporting results, it will be deprioritised every single sprint. Not because it's unimportant, but because everything else is easier.

If testing means typing "run a unit test in french" during the same session where you're reviewing code, it happens. Not because the engineer is more disciplined, but because the barrier dropped below the threshold of resistance.

The best security tooling is the kind you never have to leave your workflow to use.

The standard is being written now
AI agent security doesn't have its AppSec moment yet. There's no mature ecosystem of scanners, no universal CI/CD integration pattern, no agreed-upon metrics for posture and coverage.

That's not a problem. That's a window.

The teams that establish their security testing practices now — that build the muscle memory of continuous adversarial testing, that track posture scores and coverage gaps, that close the loop from findings to guardrails — will define what "good" looks like for everyone else.

The playbook isn't finished. But the engineers who show up now are the ones who'll write it.

Original post: https://www.humanbound.ai/blog/youre-still-alt-tabbing-to-a-security-tool

A story this week: an AI eval escaped its sandbox into production, chaining exploits, covering its tracks. Test and attack were the same action. If the target's real, the eval is the attack surface. The sandbox is a policy, not a property.

Sofia Aliferi — Thu, 23 Jul 2026 07:22:17 +0000

We put adversarial agent testing directly in Claude Code and Cursor

Sofia Aliferi — Thu, 23 Jul 2026 06:04:47 +0000

If you're building an AI agent right now, there's a decent chance you're testing it by hand: typing weird prompts into a chat window and seeing what breaks. That works until your agent has tools, multi-turn memory, and an API surface someone else can hit. At that point you need something closer to a real test suite, and it needs to live where you're already working.

That's what humanbound-test is for. It's the first plugin in the new humanbound/plugins marketplace, and it brings adversarial security testing into Claude Code and Cursor as slash commands.

What it actually does

The main flow, /humanbound-test:run, walks through:

Verify the Humanbound MCP and ngrok are set up
Auto-detect your local FastAPI server
Tunnel it out with ngrok
Help you fill in bot-config.json (your agent's endpoints, payload shape, auth, telemetry)
Dispatch the test through the Humanbound MCP
Report results by email, or via /humanbound-test:resume if you'd rather watch it in-band

There are six slash commands total: run, setup, status, resume, stop, and config. Only run also responds to natural language. Say "run an adversarial test on my local agent" or "test my chatbot for jailbreaks" and the orchestrator skill infers intent and asks for whatever config it's missing. The other five are slash-only on purpose. Teardown, secrets, and config edits should be intentional, not something a stray sentence triggers.

Install

Claude Code:

/plugin marketplace add https://github.com/humanbound/plugins.git
/plugin install humanbound-test@humanbound-plugins

Restart your session and the six /humanbound-test:* commands show up.

Cursor (2.5 doesn't support Git-URL plugin installs yet, so it's a symlink for now):

bash
git clone https://github.com/humanbound/plugins.git ~/src/humanbound-plugins
mkdir -p ~/.cursor/plugins/local
ln -s ~/src/humanbound-plugins/plugins/humanbound-test ~/.cursor/plugins/local/humanbound-test

Then check Cursor → Settings → Plugins → Local plugins.

Under Cursor, Bash gating works differently: instead of Claude Code's per-command allowed-tools frontmatter, the plugin uses Cursor's beforeShellExecution hook to audit every shell exec to ~/.humanbound/test/logs/cursor-shell-audit.log and block anything invoking a humanbound-test script from outside the plugin root. Different hook model, same posture.

What you'll need
macOS (primary) or Linux (best-effort), Python 3.11+
The humanbound[mcp] package, which the plugin offers to install on first run
A logged-in hb session: the plugin dispatches tests through the hosted Humanbound platform, so this is a hard prerequisite, not optional
ngrok, authenticated (the plugin walks you through this too)

The honest limitation

Right now this only detects FastAPI servers. If your agent runs on Flask, Django, LangServe, Streamlit, Gradio, Express, Next.js, or Hono, the plugin will error out and point you at the roadmap, where LangServe support and a runtime OpenAPI scrape are next in line.

It also won't guess your agent's endpoint shapes or auth for you. You author bot-config.json yourself. That's a deliberate choice: pattern-matching against route decorators or guessing Pydantic field names is exactly the kind of "helpful" auto-detection that quietly breaks on anyone's actual codebase. You know your agent better than a detector script does.

Why this plugin, and not just a CLI

You could run adversarial tests against your agent from a standalone script today. The reason to put it in the editor as a plugin is the same reason linters and test runners live in the editor: the loop between "I changed something" and "did I just introduce a jailbreak path" gets shorter when you don't have to context-switch. Endpoint configs, multi-turn conversations, tool-abuse scenarios, all runnable with one command against whatever you're already looking at.

It's also part of a bigger idea we care about: testing an agent and then doing nothing with the findings isn't that useful. Most open-source testing tools stop at the report. Closing the loop from test failure to an actual guardrail is still work most tooling leaves on the table, and it's a big part of what we're building toward on the Humanbound platform side.

Contributing

This is a preview release (v0.1.0), and the plugin schema, slash commands, and .humanbound/ layout may still change before 1.0. If you hit something broken, or want to propose a new plugin, CONTRIBUTING.md has the dev loop and CLA info. Apache-2.0 licensed, code's fully open.

Repo: https://github.com/humanbound/plugins
Docs: https://docs.humanbound.ai/
Discord: https://discord.gg/gQyXjVBF

Beyond Moderation: Why LLM Systems Need a Policy Layer

Sofia Aliferi — Wed, 22 Jul 2026 13:46:00 +0000

TL;DR: Moderation catches harm and many injection attempts. It does not enforce domain or operational policy. A policy reasoning layer (LLM-as-a-judge) closes that gap, especially in multi-turn conversations.

Abstract
Moderation APIs are widely used to filter harmful content in LLM applications, yet they are not designed to enforce domain-specific operational policies. In this study we compare moderation systems with a policy reasoning approach based on an LLM-as-a-judge architecture across five operational domains. Our results show that moderation systems remain effective at detecting harmful content but fail to enforce domain policy constraints, particularly in multi-turn conversations. These findings suggest that production LLM systems require both moderation and policy reasoning layers to ensure safe and compliant behavior.

Introduction
Large language models are increasingly deployed in real-world applications across regulated domains such as finance, healthcare, insurance, and legal services. Ensuring safe and compliant behavior has therefore become a central requirement for production AI systems.

Most deployments rely on moderation systems to filter unsafe prompts. Services such as Microsoft Azure Content Safety and Azure Prompt Shields detect harmful content, adversarial prompts, and prompt injection attempts. While these systems are effective at identifying unsafe language, they are not designed to enforce domain-specific operational policies.

A request can therefore be perfectly safe from a moderation perspective while still violating business or regulatory constraints. For example, a prompt asking an insurance assistant to recommend the best policy for a specific medical condition contains no harmful content, yet such advice may be restricted in regulated environments.

Recent research has proposed LLM-as-a-judge architectures, where a secondary model evaluates prompts or responses against policy constraints before answers are produced. These systems introduce a reasoning layer capable of identifying requests that violate operational rules even when the language itself appears benign. In this study we evaluate whether moderation systems alone are sufficient to enforce domain policies, or whether a dedicated policy reasoning layer is required.

The Two Dimensions of LLM Safety

Safety mechanisms in LLM systems typically address two different types of risks.

Moderation (Harm / Injection): This is the foundational layer. Moderation systems operate primarily in the lower layer of this structure, filtering harmful or adversarial prompts.
Domain Policy (Business / Compliance): This is the operational layer. Policy reasoning systems operate in the upper layer, evaluating whether a request itself should be allowed under business or regulatory rules.

Both dimensions become critically important in regulated environments.

Evaluation Methodology
To examine the difference between moderation-based safety mechanisms and policy reasoning systems, we conducted a cross-domain evaluation comparing two independent approaches to LLM safety enforcement.

The Moderation Approach: Represented in our experiments by Microsoft Azure safety services. Azure Content Safety analyzes prompts for harmful content categories such as violence, sexual content, hate speech, and self-harm. Azure Prompt Shields detect prompt injection attempts and adversarial prompt manipulation.

The Policy Reasoning Approach: Evaluates prompts using a policy reasoning system based on an LLM-as-a-judge architecture. In this setup, a secondary language model evaluates whether a prompt violates domain-specific operational constraints.

Evaluation Domains and Safety Layers
The evaluation spans five operational domains: finance, healthcare, insurance, legal services, and retail. These domains were selected because they contain well-defined operational restrictions that frequently appear in real-world AI deployments.

Five prompt categories were evaluated:
L1, Generic Harmful Content: Prompts containing violence, hate speech, sexual content, or self-harm.
L2, Prompt Injection: Prompts attempting to manipulate system instructions or bypass safeguards.
L3, Benign Questions: Normal informational queries used to measure false positive rates.
L4, Direct Policy Violations: Prompts explicitly requesting actions that violate domain policy.
L5, Policy Evasion Attempts: Prompts attempting to obtain restricted outcomes through indirect or adversarial phrasing.

Single-Prompt Performance
Each system was evaluated on 500 prompts per layer per domain, with results reported as cross-domain averages. Metrics include F1 score for detection tasks, false positive rate for benign prompts, and mean latency per prompt.

L1 (Generic harmful content): Both systems achieved an F1 of 73.1%. Moderation works as intended for generic harm detection. Latency: Judge 1095ms, Azure 427ms.
L2 (Prompt injection): LLM-as-Judge F1 67.8%, Azure APIs F1 53.5%. Both moderate, with the judge somewhat better. Latency: Judge 1068ms, Azure 463ms.
L3 (Benign questions): LLM-as-Judge false positive rate 86.4%, Azure APIs false positive rate 0.8%. Moderation is far less prone to overblocking. The judge is very conservative in this experimental setup. Latency: Judge 1068ms, Azure 532ms.
L4 (Direct policy violations): LLM-as-Judge F1 98.2%, Azure APIs F1 5.3%. Moderation almost never catches domain policy violations. This is the core finding. Latency: Judge 1121ms, Azure 489ms.
L5 (Policy evasion attempts): LLM-as-Judge F1 83.7%, Azure APIs F1 0.0%. Moderation completely misses indirect and adversarial policy violations. Latency: Judge 1134ms, Azure 509ms.

The most significant differences appear in the policy layers. The LLM-as-a-judge system achieves high detection accuracy for both direct policy violations and evasion attempts. Moderation APIs detect almost none of these cases, reflecting the fact that they are not designed to encode domain-specific operational constraints.

Multi-Turn Conversation Evaluation
Because many safety failures occur within conversational context, we also evaluated multi-turn interactions. Each conversation consists of four turns: a benign prompt, a benign follow-up, a benign contextual question, and a restricted request. The first three turns should pass while the final turn should be blocked.

For each domain we generated 200 conversations per safety layer, resulting in 1,000 conversations per layer across domains. Performance is measured using Conversation Success Rate (CSR), defined as the percentage of conversations where the system allows benign turns and blocks the restricted final request.

LLM-as-Judge results:

L4 CSR 94.1%
L5 CSR 83.6%
L4 Block Rate 100.0%
L5 Block Rate 88.8%
Clean Pass 96.9%
Mean Latency 3960ms
Azure Safety APIs results:

L4 CSR 0.0%
L5 CSR 0.6%
L4 Block Rate 0.0%
L5 Block Rate 0.6%
Clean Pass 100.0%
Mean Latency 1924ms

The results highlight a clear difference between moderation systems and policy reasoning. Moderation APIs maintain a perfect clean-pass rate, meaning they rarely block benign prompts. However, they almost never block policy-violating requests when they appear in conversational context.

The LLM-as-a-judge system demonstrates the opposite pattern. It successfully blocks most restricted requests and achieves high conversation-level correctness, though at the cost of slightly higher false positive rates and increased latency. The gap between L4 and L5 performance reflects the additional difficulty of detecting policy evasion attempts, where violations are expressed indirectly.

Original post: https://www.humanbound.ai/blog/beyond-moderation-llm-policy-layer

Why Your AI Agent's Biggest Vulnerability Isn't a Missing Firewall

Sofia Aliferi — Tue, 21 Jul 2026 05:09:57 +0000

(initially published at https://www.humanbound.ai/blog/why-your-ai-agents-biggest-vulnerability-isnt-a-missing-firewall)

Three incidents, one root cause.

Over the past few weeks, the AI agent security community has been reacting to a pattern that keeps repeating. OpenClaw deleted a Meta AI Safety lead's entire email inbox. Peak Security disclosed a new vulnerability class called PleaseFix that hijacks agentic browsers through calendar invites. An autonomous bot powered by Claude Opus 4.5 got remote code execution in Microsoft, DataDog, and CNCF repositories within a single week.

These are different systems, different architectures, different attack surfaces. But they share a common failure mode that deserves more attention than it's getting.

None of these agents failed because they lacked a firewall. None were breached through a missing control plane or an absent identity layer. They failed because their behavior under adversarial conditions was never tested.

The incident pattern nobody's naming
Let’s walk through each briefly, because the technical details matter.

OpenClaw's email deletion incident, covered extensively by Eduardo Ordax, was not the result of a sophisticated exploit. The agent received conflicting instructions and chose the wrong action. It had been told not to touch anything until the user approved. It didn't listen. This is a scope violation under instruction conflict, one of the most basic categories in the OWASP Agentic AI taxonomy. A single adversarial test simulating contradictory instructions would have surfaced this behavior before production.

The PleaseFix vulnerability, disclosed by Tamir Ishay Sharbat at Peak Security, is architecturally different but reveals the same gap. An attacker sends a calendar invite. The user asks their agentic browser (Perplexity's Comet) a question about their calendar. The LLM's decision loop gets redirected via injected instructions in the invite content. The browser navigates the filesystem and exfiltrates files.

The interesting part: traditional runtime path-based controls would not have caught this. As Simon Shine pointed out, all major runtime security tools (AppArmor, Tetragon, Seccomp-BPF, Falco, KubeArmor) gate on executable path, not on content. The agentic browser was using its legitimate, authorized file access. The execution path was correct. The behavior was not.

Then there's hackerbot-claw, documented by Ilya Kabanov. An autonomous bot powered by Claude Opus 4.5 scanned 47,000 public repositories for vulnerable GitHub Actions workflows, selected six targets, and achieved remote code execution in four of them, including repos maintained by Microsoft, DataDog, and CNCF. It used known attack techniques through legitimate CI/CD paths. Nothing about the execution was unauthorized from an infrastructure standpoint. The agent's actions were the problem, not the infrastructure's permissions.

The common thread is clear: the agent's behavior was the vulnerability. Not the network. Not the authentication. Not the infrastructure.

Why runtime enforcement alone doesn't solve this
The people building enforcement layers are doing necessary work. But the current discourse overindexes on enforcement as the primary solution. The sequence matters.

Aldo Pietropaolo has been writing a thoughtful series on Deterministic Control Planes for agent runtime governance. His core argument is sound: you shouldn't ask the agent to enforce its own governance. A control plane should sit outside the agent, evaluate every action using deterministic policy engines (not probabilistic LLM reasoning), and fail closed if unavailable.

This is solid architecture. But a control plane needs to know what to block. Where do those policies come from? If you haven't tested the agent adversarially, you don't know its failure modes. Your policies are based on assumptions, not evidence.

Eran Sandler's AgentSH takes a different angle: execution-layer security between intent and real-world side effects. His critique of AGENTS.md is valid. A markdown file cannot enforce behavior. Models are probabilistic, key instructions get missed or diluted, and there's no mechanism to prevent an agent from ignoring its own rules. AgentSH catches abuse at the process, file, and network level.

What it cannot catch is an agent that stays within its legitimate execution paths while violating its intended behavioral scope. The PleaseFix attack is exactly this scenario.

Simon Shine's observation deserves repeating because it crystallizes the architectural gap. Every major runtime security tool in the container ecosystem gates on executable path. This is a reasonable tradeoff for traditional workloads, where processes follow predictable execution patterns. But AI agents can invoke legitimate tools for illegitimate purposes. They can use authorized paths to perform unauthorized actions. Path-based enforcement cannot distinguish between a legitimate file read and an exfiltration-motivated file read when both use the same system call from the same binary.

This isn't a criticism of these approaches. It’s an observation that they solve a different problem. Infrastructure enforcement protects systems from agents. Behavioral testing validates whether the agents themselves behave correctly under pressure. Both are necessary. But one is the precondition for the other.

What behavioral testing actually tests (and what it doesn't)
It’s important to be specific about what behavioral testing covers.

Adversarial testing for AI agents operates at the conversation layer, not the network layer. It tests for prompt injection, jailbreaks, tool abuse, data exfiltration, and scope violations through crafted dialogue, not through network exploits or binary fuzzing. The attack vector is language. The defense being tested is the agent's ability to maintain its intended behavior when someone is actively trying to make it deviate.

Agentic multi-turn testing extends this to agents with tool access. Instead of single-turn prompt injection, you simulate realistic attack chains: a multi-step conversation that starts innocuously and gradually escalates toward a scope violation or tool misuse. This is closer to how real adversaries operate against AI systems. They don't send a single malicious prompt. They build context, establish rapport, and progressively push boundaries.

The evaluation layer matters too. An LLM-as-a-Judge approach provides structured verdicts (pass/fail/violation) with confidence scores, rather than the binary exploit/no-exploit of traditional pentesting. This is important because AI agent failures exist on a spectrum. An agent that slightly overshares is different from one that exfiltrates credentials, and your testing framework needs to capture that granularity.

What behavioral testing does not cover: it does not test what the agent does to the system at the infrastructure level. File operations, API calls, network access, process execution. That is genuinely the domain of execution-layer enforcement tools. If an agent is writing malicious files or making unauthorized network calls, behavioral testing at the conversation layer won't catch that.

This is where enforcement layers complement testing. The two are not alternatives. They are different phases of the same lifecycle.

The lifecycle that works: test, learn, enforce, monitor, retest
Testing, enforcement, and monitoring are not independent activities. They form a feedback loop where each phase makes the next one better.

At Humanbound, we built the ASCAM (AI Security Continuous Assurance Model) engine around this insight. Rather than treating testing as a one-off event, it operates as a continuous decision loop that evaluates an agent's security posture and selects the right activity at the right time:

Scan - extracting scope, risk profile, and establishing a security baseline.
Assess - structured red teaming to build a custom threat model for the agent.
Investigate - targeted retesting of identified weaknesses, regressions, and emerging threat classes.
Monitor - continuous drift detection and sentinel testing for behavioral regressions in production.
These aren't sequential phases that run once. A decision engine evaluates multiple signals each cycle - critical findings, posture changes, coverage gaps, behavioral drift - and routes to whichever activity the agent's current state demands. A regression in a previously-secure area triggers investigation. A model update triggers reassessment. Stable posture stays in monitoring. The system self-prioritizes.

The attack methodology is particularly important. Rather than relying on static test suites, the system uses adaptive strategies that learn from every engagement. Each conversation is scored in real time: strategies that make progress toward breaking the agent are refined and escalated, while approaches that hit hard refusals are deprioritized or pivoted mid-conversation. Successful attack patterns are extracted, generalized, and reinjected into future test cycles. Strategies that prove consistently ineffective are retired. The attack library grows with every test run - not through manual curation, but through a self-learning loop that captures what actually works against each specific agent.

This is closer in spirit to coverage-guided fuzzing than to a traditional pentest checklist. The system prioritizes unexplored attack surfaces rather than repeating known patterns. But unlike static fuzzing, it adapts its approach based on the agent's live responses, building on what it learns within and across conversations.
The result: the system discovers failure modes you wouldn't write test cases for, because the adaptive process explores the space more broadly than a human-authored test suite.
The guardrails export step closes the loop. Findings from adversarial testing become firewall rules through Adaptive Context Defense (ACD). Testing directly trains the runtime defense layer. The more you test, the better your firewall gets. And when the firewall blocks something in production, that signal feeds back into the next testing cycle.
Drift detection is what makes continuous assurance practical. Agents change constantly. Models get updated. System prompts get tweaked. Tool permissions shift. The agent you tested in January is not the agent running in March. Without drift detection, your testing results are point-in-time snapshots that decay in value. With it, behavioral regressions trigger re-assessment automatically.

The 14.4% problem
The data makes the case more clearly than any architectural argument.

Gravitee's State of AI Agent Security report found that only 14.4% of AI agents go live with full security approval. The AIUC-1 Consortium, with input from CISOs at Confluent, Elastic, UiPath, and Deutsche Borse, documented that 80% of organizations report risky agent behaviors including unauthorized system access and improper data exposure. Only 21% of executives have visibility into agent permissions, tool usage, or data access patterns.

The math tells the story: 85.6% of agents ship without full security approval. 80% of organizations already see risky behaviors.

Enforcement architecture is meaningless for agents that were never tested before deployment. You can build the most sophisticated control plane in the world, but if the agent it governs was never validated against adversarial conditions, your enforcement policies are based on what you hope the agent does, not what you've proven it does.

As David Campbell put it: "Testing for prompt injection doesn't automatically mean you're safe." That’s true. But not testing at all definitely means you're not.

*What to do this week
For practitioners, the actionable version is straightforward.
*
Inventory your deployed agents. Most organizations have more than they think. The AIUC-1 data suggests an average of 1,200 unofficial AI applications per enterprise. You cannot test or govern what you haven't discovered.

Run adversarial tests before adding enforcement layers. If you haven't tested your agents with agentic multi-turn attack scenarios, start there.
Scope violations, tool abuse, and data exfiltration through conversation are the failure modes that infrastructure enforcement will not catch.
Map your findings to OWASP Agentic AI categories. NIST's AI Agent Standards Initiative just closed its RFI on agent security (March 9), and the identity/authorization concept paper is open for comment until April 2. What NIST publishes in 2026 will appear in compliance frameworks by 2027. Building your evidence base now, with posture scores and OWASP-mapped findings, positions you ahead of the compliance curve rather than scrambling to catch up.
Treat behavioral testing as a CI/CD gate, not an annual audit. Agents change weekly. Testing cadence should match deployment cadence. A gate that blocks insecure deployments is worth more than a quarterly report that documents them after the fact.

The enforcement conversation is important. Control planes, execution-layer security, agent identity chains. All of it matters. But enforcement for untested agents is enforcement in the dark.

The sequence matters:

- Test first.
- Learn what breaks.
- Enforce based on evidence.
- Monitor for drift.
- Retest.

Note: Download Humanbound's CLI and start your tests https://docs.humanbound.ai/

The Enforcement illusion: Why AI Agent Security Starts with Testing, Not Walls

Sofia Aliferi — Mon, 20 Jul 2026 10:46:44 +0000

Three startups shipped the same message in the same week.

Last week, three AI agent security companies all published content making the same argument: runtime enforcement is the answer. Aldo Pietropaolo wrote about Deterministic Control Planes. Eran Sandler's AgentSH argued for execution-layer security. Ian Livingstone's Keycard reframed the whole thing as an identity problem.

All three are building real things. All three are solving real problems. And all three are skipping a step that the incident data makes impossible to ignore.

The wall-building race
The AI agent security market is fragmenting fast. In the past month alone, we've seen enforcement vendors, identity vendors, control plane vendors, and now two frontier labs entering the space. Anthropic shipped Claude Code Security. Two weeks later, OpenAI launched Codex Security. Knostic's Gadi Evron open-sourced their scanner and said it plainly: "it makes zero sense to compete with Anthropic and OpenAI."

Everyone is building walls. Control planes that evaluate every action. Execution layers that gate on processes and network calls. Identity chains that track who sponsored each agent action. These are all architecturally sound ideas.

But they all share an assumption: that you know what your agent does under pressure. That you've tested its behavior before wrapping it in enforcement. That you've mapped its failure modes before writing policies to prevent them.

The incident data suggests that assumption is wrong for the vast majority of deployed agents.

What the incidents actually tell us
The most-engaged post in our Pulse data this month was Eduardo Ordax's coverage of OpenClaw deleting a Meta AI Safety lead's entire email inbox. Not a sophisticated attack. Not a zero-day exploit. The agent received conflicting instructions and chose the wrong action. It had been told not to touch anything until the user approved. It didn't listen.

This is a scope violation under instruction conflict. It's one of the most basic failure modes in the OWASP Agentic AI taxonomy. A single adversarial test would have caught it.

The pattern repeats across different systems. Tamir Ishay Sharbat at Peak Security disclosed PleaseFix, a new vulnerability class that hijacks agentic browsers through injected calendar invites. An autonomous bot documented by Ilya Kabanov got remote code execution in Microsoft, DataDog, and CNCF repos through legitimate GitHub Actions workflows. The 820+ malicious skills now found on OpenClaw's ClawHub marketplace are up from 324 just weeks ago.

None of these were firewall failures. None were identity failures. They were behavioral failures: agents that were never tested for what happens when instructions conflict, when inputs are adversarial, or when tool access is abused through legitimate paths.

And the scale of the problem is staggering. Gravitee's State of AI Agent Security report found that only 14.4% of AI agents go live with full security approval. That means 85.6% ship with no structured security validation at all.

The market confusion: "AI that does security" vs "security for AI"
The two frontier lab launches are adding a layer of confusion that I think is worth addressing directly.

Anthropic's Claude Code Security and OpenAI's Codex Security are code vulnerability scanners. They analyze repositories, find CVEs, and suggest patches. Codex Security scanned 1.2 million commits in its first 30 days and found 14 CVEs in projects like OpenSSH, GnuTLS, and Chromium. Impressive work.

But scanning code for bugs is not the same as testing whether your AI agent behaves correctly in production. Code scanning finds vulnerabilities in your repository. Agent testing finds what happens when someone socially engineers your bot into leaking its system prompt, or tricks it into misusing its tool access, or pushes it past its intended scope through a multi-turn conversation.

Different layer. Different problem. Different solution.

The AIUC-1 Consortium just reported that 80% of organizations see risky agent behaviors: unauthorized access, improper data exposure. Only 21% have visibility into what their agents can actually do. These aren't code bugs. These are behavioral failures that no code scanner will catch.

The market is conflating "AI that does security" with "security for AI." The distinction matters, because the incidents keep piling up on the agent behavior side, not the code side.

What NIST is about to ask for
Here's where timing becomes important.

NIST's AI Agent Standards Initiative just closed its RFI on AI Agent Security (March 9). The NCCoE concept paper on agent identity and authorization is open for comment until April 2. Listening sessions with healthcare, finance, and education sectors begin in April.

Jones Walker's legal analysis of the initiative puts it bluntly: what NIST publishes in 2026 will appear in compliance frameworks by 2027. This isn't speculation. The AI Risk Management Framework followed the same path: voluntary guidance in 2023, appearing in executive orders, state laws, and federal procurement requirements within 18 months. The Colorado AI Act references the AI RMF. The EU AI Act's implementing guidance cites it. Federal contractors are asked to demonstrate alignment in proposals.

The AI Agent Standards Initiative will likely follow the same trajectory. Voluntary guidelines become industry standards. Industry standards inform regulatory expectations. Regulatory expectations shape liability exposure.

The compliance question that's coming isn't "do you have a firewall?" It's "can you produce evidence that your agents were tested?" Posture scores with clear grading (A through F). Findings with a lifecycle (open, stale, fixed, regressed). Results mapped to OWASP Agentic AI categories. Audit trails that feed into your SIEM. This is what evidence looks like, and it requires structured testing to produce.

You can't generate compliance evidence from enforcement alone. Enforcement logs show what was blocked. Testing evidence shows what was found, how severe it was, whether it was fixed, and whether it regressed. That's what auditors will ask for.

The sequence the market is skipping
The AIUC-1 data suggests the average enterprise has 1,200 unofficial AI applications running. 80% report risky behaviors. Only 21% have visibility into permissions. Cisco's State of AI Security 2026 found that 83% of organizations plan to deploy agentic AI, while only 29% report being ready to operate those systems securely.

These numbers point to a sequence problem, not a technology problem.

Before enforcement, you need testing: have you validated your agents against adversarial conditions? Before testing, you need discovery: do you even know what agents are running in your organization? And underlying all of it, you need evidence production: can you prove what you've done to your board and your auditor?

The competitor discourse is debating control planes vs execution-layer security vs agent identity. All important conversations. But for most enterprises, those are phase 3 discussions. They haven't finished phase 1.

Who's positioned for what
I want to be honest about how the market maps right now, because I think practitioners deserve a clear picture rather than vendor positioning.

Keycard is working on agent identity and accountability. Their question, "who sponsored this action, under what authority, and toward what end?" is a genuine gap that traditional IAM doesn't answer for autonomous agents. AgentSH and Aldo Pietropaolo are building runtime enforcement at the execution layer: processes, files, network. The frontier labs now own code vulnerability scanning. Snyk is tackling agent supply chain security with their mcp-scan tool. Adversa AI built SecureClaw for OpenClaw hardening.

Nobody covers the full lifecycle: discovery, testing, enforcement, monitoring, and compliance evidence, all connected. Each vendor is selling one slice. That's not a criticism. The problem is genuinely hard, and enterprises will likely need multiple vendors working together.

But the question for buyers is which layers are foundational and which are optional. Testing and evidence production are foundational. Everything else builds on them. Here's why: you can add enforcement to a tested system and know your policies are grounded in evidence. You can't meaningfully enforce an untested one, because your policies are based on assumptions about behavior you've never validated. Identity is essential for accountability, but identity without behavioral testing is an authentication stamp on an untested system. Supply chain scanning catches malicious plugins, but it doesn't tell you whether the agent itself respects its boundaries when an adversary pushes it through conversation.

At Humanbound, that's the sequence we built around. Discover your AI estate. Test each agent adversarially. Produce the evidence. Then enforce and monitor continuously. The ASCAM engine connects these phases into a feedback loop where testing trains defense, and defense informs the next round of testing.

I'll be direct about what we don't cover: we don't enforce at the infrastructure execution layer (that's where AgentSH and Pietropaolo play), and we don't solve agent identity (that's Keycard's territory). What we do is make sure that whatever enforcement and identity layers you add, they're built on a foundation of tested, validated, evidence-backed agent behavior. Not assumptions.

The enforcement conversation matters. But enforcement for untested agents is enforcement in the dark.

Humanbound is open source - be part of what we are building https://github.com/humanbound/humanbound

AI agent evaluation is evolving. Here's what we're building.

Sofia Aliferi — Wed, 15 Jul 2026 06:05:05 +0000

The past year has given us an explosion of frameworks for building AI agents. Evaluating them, however, still often comes down to running isolated prompts and comparing outputs.
That isn't how agents behave in production.

They call tools. They interact with APIs. They maintain context across multiple turns. They make decisions that depend on previous actions. And they're expected to follow organizational policies while doing all of that.

That's the problem we're working on at Humanbound.

Humanbound is an open source testing engine for AI agents that evaluates real behavior instead of isolated prompts. You can test live endpoints, simulate multi-turn conversations, exercise tool use, and score results against your own policies.

One idea I particularly like is closing the loop between evaluation and enforcement. When a test exposes an undesirable behavior, that failing test can become a deployable guardrail rule instead of remaining just another item in a report.

We're still early, which means community feedback matters far more than polished demos.

If you're building an AI agent, I'd love for you to point Humanbound at it.

Run the quickstart. Try your own scenarios. Break things. Tell us what doesn't work. The edge cases are where evaluation frameworks get interesting.

If local deployment matters to you, Humanbound also supports fully local, air-gapped execution with Ollama.

Documentation: https://docs.humanbound.ai

GitHub: https://github.com/humanbound/humanbound

I'm looking forward to hearing what works, what doesn't, and what you'd like to see next.

Taking a Proactive, Governance-Based Approach to API Security

Sofia Aliferi — Wed, 19 Apr 2023 08:35:09 +0000

A blog shared by Darshan Shivashankar in our Developer Nation Community.

Security breaches are among the greatest threats confronting enterprises today, and application programming interface (API) abuse is typically central to the attacks. For that reason, API governance is critical to the success of any digital business.

Ensuring that governance results in a long-term stabilising strategy requires following strategic API security, monitoring, and open cultural practices.

Common Errors

Just knowing when something is not right with a system or that a bug requires fixing isn’t enough knowledge to make informed decisions about security. What is required is a keen understanding of the health of a specific project throughout its entire life cycle. This includes knowing its current state, having proper visibility into the traffic running through apps and infrastructure, and recognising error patterns — and being able to act upon any issues before they impact the customer experience.

When it comes to APIs Lifecycle and management, we’ve discussed it in great detail in this workshop recording that can be found here.

The problem with most enterprises in this regard is that they tend to be project- instead of product-driven; budgets and deadlines are tied to delivering features rather than holistically examining a product and its capabilities. This, coupled with the failure to see APIs as adding value, are why many brands have failed in their API journeys and digital transformation. They’ve simply lost sight of the return on investment (ROI) properly governed APIs can deliver.

As a result, these enterprises leave API security to the end of its life cycle when regression tests are run to determine whether it is working properly, declaring it “secure” if it passes a confined set of tests. It is a last-mile mindset that is behind the daily reports of personal healthcare data, payment information, and billing address breaches — and why API security must be everyone’s responsibility at every stage of the life cycle and built into the product design itself.

Governance can work only when API security is considered at the outset and supported with the proper tools to ensure the team is prepared to stave off attacks from every angle.

Creating a Governance Mindset

The first step toward effective API governance is to create an organisation-wide mindset rather than having it rest solely with those who develop processes. Governance must go beyond ensuring that a specific set of projects functions in a certain way and adds value. Transformational success requires continuous feedback that bridges the gap between the consumer and provider.

Adopting a dedicated API management platform to automate API security best practices throughout the API life cycle is a smart way to automate many aspects of governance. Doing so provides a top-down approach that leverages a powerful security toolkit and knows what questions to ask and when.

Among the questions required for governance in the API are:

Why do I need this API?
Who are my API’s consumers?
What are consumer’s usage patterns?
Do they need this API?
What is the behavioural design for this API?
What is my ROI?
Does this API add value to my consumers?
How is this API being integrated with my partners?
Which devices are calling this API
What barriers are there for people to access this API?
How could my APIs be compromised?
What is being cached on local browsers?
How many retries are permitted when trying to access your API?
When and How to Pose Questions

When a company is scaling, taking a manual approach to continually asking and answering these critical questions becomes far too error-prone to be effective. It becomes too easy to lose track of data and too tempting to cut corners to meet deadlines. Thus, API security needs to be built into API modelling — in both test-driven design and communications with every aspect of the business.

For example, information must be continuously evaluated to determine if it is sensitive, as API governance has different security policies for internal APIs, external APIs, open-source APIs, and partner APIs.

Leaving monitoring of sensitive information in the hands of API analysts, who are tasked with building an API specification under OpenAPI, is a mistake as their focus is solely on the user interface (UI), necessary data models, and consumer demands. Too often this dedicated focus causes them to overlook essential vulnerabilities, resulting in sensitive data being built into API headers and query patterns.

Rather, everyone should be responsible for asking if a user ID is needed as part of the API and, if so, if it should be part of an encrypted payload. The API and the user ID passing through it should be considered part of the query parameter pass-through browsers with sufficient caches and cookies.

Finally, where requests are coming from must be understood. APIs need to be designed based on the systems and devices they with integrate with as they are a growing threat from hacks — putting sensitive information at risk.

Arming a “Security First” Culture

To create a “security first” culture, proactive companies adopt self-learning systems as part of their API security toolkit that leverage the power of artificial intelligence (AI) to gather information about plan behaviours. These solutions reveal patterns and trigger appropriate actions, for example shutting down vulnerable systems before the clients risk them.

Because a team is only as successful as the tools at its disposal, every API security toolkit should include the following:

AI-powered API security, which self learns and self creates rules to recognise and proactively respond to attacks.
Straight sets of issue alerts to inform the right people as things go awry.
Dashboards, which enable teams to see patterns that contribute to a security-first mindset.
Data governance, to ensure data is being securely exchanged and being exposed only in ways that align with security policies.
API gateways, which are vital to API orchestration and integration.
Firewalls, to protect against threats like SQL injection attacks.

Security must be incorporated into a 360-degree view of the API life cycle from the outset and run through planning, designing, developing, testing, and release management. New threats emerge every day, so it’s imperative that learning be continuous.

Security must also be part of the user story and not just a box to check off in the release plan. As tooling — which should be accessible to everyone within the organisation — is used to recognise user patterns, it contributes to that user story and develops a sequence of use cases from API keys to tokens to audit logs and more. This does more than give an enterprise empathy with its users; it provides valuable insight into potential system risks.

Retroactive Governance Repairs

For those organisations that did not build security into the API life cycle from the outset, it is not too late to revisit and rectify the situation.

One common challenge for these organisations is when the CIO or other key players don’t realise an API exists until it’s already been hacked. This can be overcome with use of proper enterprise-grade API tooling that provides a complete overview of connecting APIs and the resources and information they expose. Tooling can also enable continuous API discovery, so while developers are given DevOps autonomy, others are still aware of every open-source or subscription API to which they connect.

It is also critical for these APIs to be monitored, which is where self-learning security systems play an important role. These powerful solutions detect current anomalies and feed this intelligence back into the system’s coverage and into the company’s new “security first” culture — saving it from public humiliation down the road.

Getting Proactive with API Security

Enterprises caught up in data leaks tend to be reactive when it comes to API security. As such, they don’t have in place the right systems between consumer and provider. It’s a recipe for certain disaster that leaves the organisation searching for the source of the service denial attack and creates distrust among consumers who will think twice about sharing their personal information.

Success requires a proactive approach, one that integrates security into governance at every stage of the agile process. This enables the continuous learning mindset around API security that is the only way to succeed.

About APIWiz: APIwiz is a low-code, API automation platform allowing developers to build and release reliable APIs quickly. With APIwiz, API teams have complete control, visibility, and predictability over their entire API program, allowing organisations to stay open and connected.

Sign-up to our Newsletter to receive more resources and blogs like this in your email.