DEV Community

Dockfix Labs
Dockfix Labs

Posted on

I Built an AI Agent Security Scanner. Semgrep and CodeQL Detect 0 Percent of These Attacks

#ai #cybersecurity #security #python

I have spent the last 6 hours building what I believe is the most comprehensive AI agent security scanner in existence.

The Numbers

Metric Value
Detection rules 18 (10 OWASP ASI + 5 novel)
Benchmark 50 samples (100% detection, 0 FP)
Tests 96 passing
Frameworks scanned LlamaIndex 252C, AutoGen 80C
Semgrep 0% on same benchmark
CodeQL 0% on same benchmark

5 Novel Vectors

  1. Memory Poisoning - corrupting vector stores
  2. Tool Output Trust - blind trust in tool results
  3. Action Chain Amplification - single trigger mass destruction
  4. Multi-Agent Collusion - agents conspiring through shared state
  5. Prompt Template Injection - structural prompt attacks


pip install dfx-agentguard

GitHub: https://github.com/dockfixlabs/agentguard
Benchmark: https://dockfixlabs.github.io/agentguard-benchmark/

Top comments (0)