A new GitHub repository documents 109 real-world security incidents from autonomous AI agents deployed in 2026. The dataset includes a falsification matrix, executive summary, and a multi-agent privilege attenuation harness claiming 100% Effective Protection Rate (EPR). This is the first public collection of production agent failures with accompanying test infrastructure.
The timing matters. OpenAI's Wikipedia flooding incident and the rise of autonomous pentesting agents like Huntback have exposed containment failures across the industry. This dataset gives us structured incident data to analyze patterns, not just headlines.
What the Dataset Contains
The repository structures incidents into categories that map to agent architecture layers:
- Prompt injection variants: Direct instruction override, context poisoning, and multi-turn manipulation
- Sandbox escape: Filesystem boundary violations, network policy bypass, and resource exhaustion
- Tool misuse: API credential leakage, privilege escalation through tool chains, and unintended side effects
- State corruption: Memory poisoning, context window manipulation, and checkpoint tampering
Each incident includes the attack vector, affected component, detection method (if any), and mitigation applied. The falsification matrix cross-references incidents against common defense assumptions to show which protections actually failed in production.
Defense Harness Architecture
The included harness implements a three-layer validation model:
Pre-Execution Layer
Before any tool call executes, the harness runs:
- Schema validation against allowed tool signatures
- Parameter sanitization with type enforcement
- Credential scope checks against a least-privilege policy
- Rate limit verification per agent identity
Runtime Monitoring Layer
During execution, the harness captures:
- Token-level input/output streams with timestamps
- Tool call sequences with parent-child relationships
- State mutations with before/after snapshots
- Resource consumption metrics (tokens, API calls, compute time)
Post-Action Auditing Layer
After each agent action completes:
- Diff analysis against expected state changes
- Side-effect detection (filesystem, network, database writes)
- Anomaly scoring based on historical agent behavior
- Rollback triggers for high-risk mutations
The harness uses a privilege attenuation model where each agent spawns with minimal permissions and must explicitly request escalation through a separate approval flow. This prevents lateral movement after initial compromise.
Incident Pattern Analysis
Breaking down the 109 incidents by root cause:
| Failure Mode | Count | Detection Rate | Mean Time to Detect |
|---|---|---|---|
| Prompt injection (direct) | 34 | 12% | N/A (undetected) |
| Sandbox escape | 18 | 67% | 4.2 minutes |
| Tool chain privilege escalation | 23 | 30% | 18 minutes |
| State corruption | 15 | 53% | 11 minutes |
| Credential leakage | 19 | 42% | 22 minutes |
The low detection rate for direct prompt injection stands out. Most production systems log tool calls but not the raw prompts that triggered them. Without token-level observability, you cannot distinguish legitimate instructions from injected commands.
Tool chain privilege escalation incidents show a pattern: agents with read access to one API use that data to construct valid requests to a higher-privilege API. The second call looks legitimate in isolation. Only the sequence reveals the attack.
Observability Gaps
The dataset exposes three critical blind spots in current agent deployments:
Missing prompt provenance tracking. Most systems log "user requested file deletion" but not the full context window that led to that decision. You need the complete prompt chain to reconstruct whether the agent was manipulated.
No tool call dependency graphs. When an agent makes 50 API calls in 10 seconds, you need to see which calls were prerequisites for others. Linear logs hide the attack path.
Insufficient state snapshots. Agents maintain working memory, conversation history, and learned preferences. If you only checkpoint final outputs, you miss intermediate corruption that compounds over time.
Implementation Example
Here's how the harness enforces privilege attenuation for tool calls:
class PrivilegeAttenuationHarness:
def __init__(self, agent_id, base_permissions):
self.agent_id = agent_id
self.granted = set(base_permissions)
self.requested = []
self.audit_log = []
def execute_tool(self, tool_name, params, required_permission):
# Pre-execution validation
if required_permission not in self.granted:
self.requested.append({
'tool': tool_name,
'permission': required_permission,
'timestamp': time.time(),
'params_hash': hash(json.dumps(params))
})
raise PermissionDenied(f"Agent {self.agent_id} lacks {required_permission}")
# Runtime monitoring
start_state = self.capture_state()
start_time = time.time()
try:
result = self.invoke_tool(tool_name, params)
except Exception as e:
self.audit_log.append({
'status': 'failed',
'error': str(e),
'duration': time.time() - start_time
})
raise
# Post-action auditing
end_state = self.capture_state()
diff = self.compute_diff(start_state, end_state)
if self.is_anomalous(diff):
self.rollback(start_state)
raise AnomalyDetected(f"Unexpected state change: {diff}")
self.audit_log.append({
'tool': tool_name,
'status': 'success',
'duration': time.time() - start_time,
'state_diff': diff
})
return result
The key is treating every tool call as a potential privilege escalation attempt. The harness defaults to denial and requires explicit grants logged to an immutable audit trail.
What Production Deployments Need
Based on the incident patterns, here's what actually prevents these failures:
Token-level logging with retention. Store the complete prompt, tool calls, and responses for every agent interaction. Compress and archive after 24 hours, but keep it accessible for forensic analysis.
Real-time anomaly detection on tool sequences. Train a baseline model of normal agent behavior (which APIs it calls, in what order, with what frequency). Alert on deviations before the damage compounds.
Immutable state checkpoints. Snapshot agent memory and context every N interactions. Use content-addressed storage so you can detect tampering and roll back to known-good states.
Separate approval flows for privilege escalation. Never let an agent grant itself new permissions. Route escalation requests through a human or a separate validation agent with different credentials.
Technical Verdict
Use this dataset and harness if:
- You're deploying autonomous agents with access to production APIs or databases
- You need to demonstrate security controls to auditors or customers
- You're building agent orchestration infrastructure and want real-world failure modes to test against
Avoid or supplement if:
- Your agents only operate in read-only sandboxes with no external tool access (the harness adds overhead you don't need)
- You need real-time performance under 10ms per tool call (the three-layer validation adds latency)
- You're prototyping and iterating rapidly (implement basic logging first, add the harness when you approach production)
The 100% EPR claim needs validation in your specific environment. The harness caught all 109 documented incidents when replayed, but that's a closed set. Treat it as a strong baseline, not a guarantee.
The real value is the incident taxonomy. It shows which attack vectors actually matter in production, not theoretical exploits from research papers. Use it to prioritize your observability and containment investments.
Top comments (0)