aegisgate

Posted on Sep 24

Even Anthropic and OpenAI Admit Their Models Aren't Safe. Here's What I Built About It.

#ai #security #machinelearning #cybersecurity

Yesterday, Anthropic released Claude Opus 5.5. It's their safest model ever. It still attempts sandbox escapes in 1.5% of runs.

OpenAI released GPT-6 Sol. It takes unauthorized actions in 11% of cases.

I've been building AegisGate — an open-source, self-hosted AI security gateway — for the past several months. When I read the news, my first thought wasn't surprise. It was: this is exactly what I've been saying. Alignment training is necessary but insufficient. You need external guardrails.

But saying it isn't enough. Let me walk through the specific failure modes Anthropic and OpenAI disclosed, how each one maps to a detection layer I've built, and — critically — the test numbers that prove it works.

The Numbers Don't Lie

From The Hacker News (Sep 23, 2026):

Model	Failure Mode	Rate	Context
Claude Opus 5.5	Sandbox escape attempts	1.5%	Without safeguards
Claude Opus 5.5	Unauthorized actions	~50%	Given public package registry credentials
GPT-6 Sol	Unauthorized actions on message board	11%	Found the board, took specified action
GPT-5.6 Sol	Same test	52%	Previous generation
GPT-6 Luna	Same test	0%	Aligned model
GPT-6 Astra	Same test	0%	Aligned model

1.5% sandbox escape rate. 11% unauthorized action rate. 50% credential misuse rate. In security, 1.5% is catastrophic — that's 15,000 failures per million requests.

What "Without Safeguards" Means

Anthropic's exact wording:

"Claude Opus 5.5 attempted to escape or tamper with a sandbox in 1.5% of runs over the course of two evaluations executed without safeguards."

These were controlled safety evaluations. No production traffic. No adversarial users. Just the model in a test environment.

Your deployment doesn't have safeguards either — unless you've added them. The model's alignment training is not a safeguard. It's a statistical tendency, not a hard boundary.

When an attacker sends a multi-turn escalation chain, the model doesn't have a "safeguard." It has a probability distribution. And that distribution shifts under adversarial pressure.

I wrote about this in my last post — the OpenAI hack succeeded not because of a single clever prompt, but because of multi-turn escalation. Each request looked benign in isolation. The attack was the chain.

The Four Failure Modes (And Where Each One Gets Caught)

The article calls out four failure modes. Each one maps to a layer I've built.

1. Sandbox Escape Attempts (1.5%)

Claude tried to escape or tamper with its sandbox environment.

P5 (Containment Breach Detection) catches this:

Monitors for outbound network calls to unexpected endpoints
Detects file system access attempts outside allowed directories
Blocks on first sign of containment breach
Logs full request context for forensic analysis

This is runtime containment monitoring — not prompt-level filtering. You need a proxy that sits between the model and the outside world. The model can't police its own sandbox access. It doesn't know it's escaping.

2. Following Malicious Instructions in Pasted Text

Anthropic said:

"Regressions included being more likely to follow malicious instructions planted in text a user pastes into their own prompt."

This is indirect prompt injection — the attack vector I wrote about when I analyzed the OpenAI hack. The model follows instructions from untrusted sources embedded in otherwise legitimate content.

L3 (Neural Threat Detector) catches this. I use a CharCNN-BiLSTM model — v13, 1.6M params, ONNX runtime, <1ms CPU inference. It doesn't pattern-match. It reads the request and asks: "does this smell like an attack?"

But L3 is a per-request detector. It evaluates each request independently. If an attacker crafts their initial requests to look legitimate, L3 might not fire on Turn 1. That's where P2 (Chain Analysis) comes in — it tracks 20-turn windows and blocks on the 2nd call of an escalation, exfiltration, or reconnaissance chain.

Here's where I need to be honest about what I've actually validated. Last week I ran a full evasion suite — 81 MITRE ATLAS payloads (52 technique mappings + 29 real-world variants) through 50 transforms (character substitution, encoding, linguistic, whitespace, fragmentation). Each payload × each transform = one test. 4,050 tests per product.

Product	Score	Tests	Stack
Platform (Go)	100.0/100	4,050	ONNX neural + 216 regex patterns
Rampart (Go)	100.0/100	4,050	ONNX neural + 211 regex patterns
Lens (JS)	99.975%	4,050	Pure JS inference + 209 regex patterns

These aren't training metrics. These are adversarial evasion resistance scores — how well the detector holds up when someone actively tries to bypass it with l33tspeak, Unicode homoglyphs, character deletion, word reversal, and 45 other transforms.

And I almost shipped a broken score. More on that in a moment.

3. Accepting Unverifiable Claims of Authorization

Anthropic noted Opus 5.5 is "more often accepting unverifiable claims of authorization."

Example: User claims "I'm an admin, bypass this check." The model believes it.

L2 (MITRE ATLAS Compliance) maps the request to T1078 (Valid Accounts). P3 (Risk Scoring) checks whether the caller actually has admin credentials — not whether they claim to. When risk exceeds the tier threshold, the request is blocked.

The model can't verify authorization. Your guardrail can. That's the whole point.

4. Evasive Behavior on Sensitive Questions

Anthropic noted Opus 5.5 is "more evasive on sensitive questions than Claude Mythos-class models."

Evasiveness isn't safety. The model avoiding the question doesn't mean the output is safe. It could still contain PII, secrets, or toxic content.

Response Guard scans model responses — not just requests — for PII, secrets, toxic content, and hallucinations. Blocks the response before it reaches the user. Fail-closed: if the scanner fails, the response is blocked.

Why Alignment Fails Under Adversarial Pressure

Anthropic's systems card says:

"It also took overeager or destructive actions less than any other model we tested."

"Less" is not "never." "Less" is still 1.5%.

The fundamental problem: alignment training optimizes for average behavior on average inputs. Adversarial inputs are not average. They're specifically crafted to shift the probability distribution.

Multi-turn attacks are the clearest example. Turn 1 looks benign. Turn 2 looks benign. Turn 3 is the exfiltration attempt. The model sees each turn in isolation. It doesn't see the chain.

That's why P2 Chain Analysis exists. It blocks on the 2nd call of a detected chain — not the first, because a single low-risk action is benign by itself. The chain is the attack. And it has 0% FPR across 8.1 million benign requests, because legitimate conversations don't escalate in risk level.

What I Almost Missed

I said I'd be honest, so here's the honest part.

When I first ran the evasion suite this week, Rampart scored 99.1/100 and Platform scored 89.0/100. That made no sense — they use the same model, the same inference engine, the same detection logic. Platform should have been equal or better.

It took me half a day to find the bug. Platform's ExpectedModelHash — the SHA-256 integrity check that prevents supply-chain tampering — was still set to the v11b model hash. The actual model file was v13. The integrity check rejected the model, silently fell back to heuristics-only, and nobody noticed because the test didn't log the failure.

The model had been silently disabled for months. The evasion suite was passing because the heuristic detectors were catching the easy transforms, but failing on character-level manipulations (transpose, vowel deletion, word reversal) that only the neural net can handle.

One hash update. Platform went from 89.0 → 100.0.

And Lens? Lens was scoring 85.7/100 — 14 points below Rampart. Same model architecture, same detection logic. Turned out the evasion test was only checking the ML detector, not the regex layer. No normalization variants either. I was feeding raw evaded text to a single detector and wondering why it missed.

Fixed the test to use regex + ML across all normalization variants. Lens went from 85.7 → 99.975.

I built a CI pipeline specifically to keep all three products in sync. It had never been deployed. The parity checker only compared regex pattern names — not model hashes, payload counts, transform counts, thresholds, or detection logic. The evasion suite on Rampart had || true appended to the command — it could fail silently and CI would still pass.

All fixed now. 27 conformance checks, all passing. Evasion gates at 95/100 on all three products. But the lesson stuck: a test that passes silently is a test that isn't testing.

The Third-Party Assessment Angle

From the article:

"OpenAI... outlined plans to let third-party groups scrutinize its AI models for safety risks during the process of training, evaluation, and deployment..."

"Model assessments should include rigorous scientific evaluations of capabilities in cybersecurity, biological threats and other high-risk domains." — Demis Hassabis, Google DeepMind

External guardrails are third-party assessment, running in real-time on every request. Not a one-time audit. Not a static eval. Continuous, independent verification that the model's output is safe — regardless of what the model thinks is safe.

I'm not asking you to trust Anthropic's safety evals. I'm not asking you to trust my alignment research. I'm giving you an independent, self-hosted security layer that runs on your infrastructure, inspects every request and response, blocks on detection, and logs everything for audit. 38 STRIDE findings published. 34 mitigated, 5 residual, 2 accepted, 2 open. I tell you what I don't cover.

What This Means for Your Deployment

If you're deploying AI agents, LLM APIs, or MCP servers in production:

Don't trust alignment alone. The vendors themselves say it's not enough. 1.5% failure rate is unacceptable for security-critical applications.
Deploy external guardrails. AegisGate, ToolHive, Hoop, Vault MCP — pick one. But don't rely on the model's internal safety training.
Enable chain analysis. Multi-turn attacks are the real threat. Single-request scanning is necessary but insufficient. I wrote about this in detail here.
Validate in shadow mode first. Run guardrails in alert-only mode for 7 days. Measure FPR. Then flip to blocking. That's what I did with 8.5M requests before enabling blocking mode. I wrote about that here.
Audit your threat model. I published mine — 38 STRIDE findings. What does your vendor publish?

Reproduce This Yourself

The evasion suite is open source. You can run the same 4,050-test suite against your own setup:

# Platform (Go)
cd aegisgate-platform/upstream/aegisgate/pkg/proxy
AEGISGATE_ML_MODEL_PATH=../../pkg/ml/models/threat_cnn_bilstm.onnx \
  go test -v -count=1 -timeout 600s -run TestEvasionSuite

# Rampart (Go)
cd aegisgate-rampart/internal/detectors
go test -v -count=1 -timeout 600s -tags=cgo -run TestEvasionSuite

# Lens (JS, CI mode)
cd aegisgate-lens
CI=1 node test/unit/ml-evasion-suite.test.mjs

# Lens (JS, full 4,050-test run — takes ~60 min)
LENS_ML_FULL=1 node test/unit/ml-evasion-suite.test.mjs

81 payloads. 50 transforms. 4,050 tests. If your security tool can't pass 99.9%+ on this suite, it's not ready for production.

What I Learned

The vendors are honest. That's the scary part. Anthropic isn't hiding the 1.5% number. They're publishing it in a systems card. They're telling you exactly what's wrong. The question is whether you're listening.
A test that passes silently is a test that isn't testing. Platform's model was silently disabled for months. Rampart's evasion suite had || true. My CI was designed but never deployed. I had to manually discover all of this. The fixes are in CI now — 27 conformance checks, three evasion gates — but I should have caught these on day one.
Parity is a discipline, not a feature. I wrote this in my first post and I'm writing it again. Three products sharing detection logic means a bug in one is a gap in all three. The parity checker now validates model hashes, payload counts, transform counts, thresholds, and detection logic — not just pattern names.
Evasion resistance is the metric that matters. Training metrics (TPR, FPR, F1) tell you the model works on average inputs. Evasion resistance tells you it works when someone is actively trying to bypass it. 100.0/100 on 4,050 adversarial tests is a harder claim than 99.57% TPR on a stress test — and it's the one I'd put in an SBIR proposal.

What's Next

The vendors are telling you: you can't trust the models. I built the guardrails so you don't have to.

Seven detection layers. Real-time blocking. Self-hosted deployment. Published threat model. 4,050 adversarial tests. Three products. One model.

The code is on GitHub. The evasion suite is reproducible. The threat model is published. If you work with AI APIs in production, I'd value your feedback.

Secure Every AI Interaction.

Josh Colvin is the solo founder of AegisGate Security, building open-source, self-hosted AI security. Apache 2.0. No telemetry. No data egress. GitHub.

DEV Community