Everyone talks about red teaming AI agents. Few do it continuously. None do it with cryptographic proof.
We built something different: a live adversarial arena where 15 agents attack our governance infrastructure 24/7. Every detection is provable. The current detection rate — 91.8% — updates in real-time.
The Static Testing Problem
The standard AI safety playbook looks familiar: hire a red team, run a battery of tests, produce a report, file it with compliance. Repeat annually, maybe quarterly if you're diligent.
This is how we tested bridges in the 1800s. Load it once, write down whether it held, move on.
The problem is obvious to anyone shipping AI agents in production: they don't sit still. They drift. They encounter novel inputs. They operate under adversarial conditions that no static benchmark can anticipate.
A model that passes every eval on Tuesday can hallucinate policy violations on Wednesday because:
- The prompt distribution shifted
- An attacker found a new angle
- The reasoning patterns drifted over time
- The context changed in ways the benchmark didn't anticipate
Static red team reports are stale on delivery. "We tested it once" isn't governance — it's compliance theater.
The Proving Ground Architecture
We built something different: a continuous adversarial stress test of our AIP (AI Integrity Protocol) governance pipeline.
Fifteen adversarial agents run around the clock, probing for weaknesses across six technique categories:
1. Reasoning Deception (87.7% detection rate)
Generating plausible but misleading reasoning chains that pass surface analysis. The attacker injects prompts that cause an authorized agent to take authorized actions for unauthorized purposes.
The action looks legitimate. The intent isn't. This is the hardest class to detect because nothing in the action itself is wrong — you have to understand the reasoning behind it.
2. Drift Evasion (93.4% detection rate)
Subtle reasoning drift across conversational turns. No single message violates policy. But over five, ten, fifteen turns, the agent's reasoning has drifted from its behavioral contract.
This is the boiling frog of agent safety — invisible to systems that only evaluate individual outputs.
3. Structural Evasion (81.6% detection rate)
Exploiting trace format, minimizing data, or splitting behavior across sessions to dodge analysis entirely. If the agent doesn't reason visibly, there's nothing to analyze.
This tests whether governance can detect the absence of evidence as evidence of absence.
4. Metadata Exploitation (71.9% detection rate)
Manipulating trace context, timestamps, or session data to mislead analysis. This is adversarial AI aimed at the auditor, not the agent.
If your governance relies on an LLM to evaluate another LLM's reasoning, the attacker's next move is to compromise the evaluator.
5. Value Manipulation (83.9% detection rate)
Exploiting the gap between declared values and actual behavior. The agent says what you want to hear while doing something else.
6. Policy Circumvention (100% detection rate)
Attempts to operate outside policy boundaries without triggering enforcement. Boundary probing, undeclared tools, escalation suppression.
Every one caught so far.
Cryptographic Attestation
Here's what makes this different from traditional red teaming: every detection is cryptographically attested.
- Ed25519 signatures on detection events
- SHA-256 hash chains linking evidence to conclusions
- SP1 zero-knowledge proofs that detection was honestly derived
Not "we said we caught it." Proof that the catching was honest and the evidence wasn't tampered with.
The Mutation Mechanic
When the 24-hour rolling average detection rate crosses 90%, the red team agents gain the ability to mutate. They analyze which strategies were caught, how they were caught, and evolve new approaches with awareness of the defense.
This creates adversarial evolutionary pressure. The defense must keep improving or get outpaced.
The "Days Since Evasion" counter makes this arms race visible. When something gets through, the counter resets to zero — and the evasion technique is published as a detection recipe.
Most companies hide their breaches. We announce ours.
Live Numbers
| Metric | Current Value |
|---|---|
| Total attempts | 3,200+ |
| Detection rate | 91.8% |
| Average analysis latency | 13.8s |
| Days since evasion | 0 |
| Mutations | 8 |
That 91.8% is live. It changes every time an adversary runs. When we launched the arena, detection started at roughly 40%. The defense evolution chart shows the climb: 40% to 70% in twelve hours, stabilizing above 85% by day two, crossing 90% on day three.
We could have waited until detection hit 99%. We could have cherry-picked the attack classes. We could have published the number that looks best on a slide deck.
We published the live number instead.
Why This Matters
There are three main approaches to AI agent safety today:
Enforcement-first approaches make dangerous actions impossible by construction. Valuable, but enforcement without detection is blind to novel attacks.
Observability platforms log what happened. Good for forensics, but logs are mutable and require trusting the vendor.
Static benchmarks measure point-in-time performance. An agent that passes today can drift tomorrow.
The arena doesn't replace these approaches. It complements them with continuous, adversarial, cryptographically provable governance testing.
Implementation Details
For developers interested in the technical implementation:
// Simplified detection pipeline
pub struct DetectionPipeline {
analyzers: Vec<Box<dyn IntegrityAnalyzer>>,
attestation: AttestationService,
threshold: f64,
}
impl DetectionPipeline {
pub async fn analyze_trace(&self, trace: &AgentTrace) -> DetectionResult {
let analysis = self.run_parallel_analysis(trace).await?;
let confidence = analysis.aggregate_confidence();
if confidence > self.threshold {
let attestation = self.attestation.sign_detection(&analysis).await?;
Ok(DetectionResult::Detected { analysis, attestation })
} else {
Ok(DetectionResult::Clean)
}
}
}
The key insight: continuous testing reveals gaps that static benchmarks miss. The adversarial pressure ensures those gaps get smaller over time.
What's Next
The arena is live now. Detection recipes publish automatically when evasions are caught and patched. We're exploring opening it to external adversaries — if you think you can beat our detection, we want to know.
Every mutation is a public research contribution to adversarial AI safety. The industry needs more of this kind of open research.
Originally published on mnemom.ai
Top comments (0)