DEV Community

Cover image for IPI-Scanner: Detecting Indirect Prompt Injection Attacks Before Your LLM Reads Them
Amit Gupta
Amit Gupta

Posted on

IPI-Scanner: Detecting Indirect Prompt Injection Attacks Before Your LLM Reads Them

#security #devops #ai #opensource

IPI-ScannerAn open-source security tool for RAG pipelines and agentic AI systems

The Problem: The Silent Attack Vector

You've probably heard about prompt injection attacks. But here's what most people don't realize: 80% of prompt injection attacks are indirect.

They don't target your prompt. They target your data.

An attacker poisons a document that your RAG system later retrieves. When your LLM reads it, hidden instructions execute silently. No alerts. No warnings. Just compromised output.

Real Examples

  • EchoLeak: Malicious email to a Copilot user leaked passwords via invisible instructions
  • HashJack: URL fragments with hidden instructions steered AI summaries
  • Perplexity Comet: Reddit posts with invisible text exfiltrated user data
  • CVE-2025-53773: GitHub Copilot RCE via PR description injection

The cost? $2.3 billion in global losses (2025). OWASP lists prompt injection as the #1 vulnerability in LLM systems.

The Solution: IPI-Scanner

I built IPI-Scanner – an open-source tool that detects indirect prompt injection attacks in documents before they reach your LLM.

Think of it as a security checkpoint for your data pipeline.

pip install ipi-scanner
ipi-scan document.pdf
Enter fullscreen mode Exit fullscreen mode

How It Works

IPI-Scanner uses a 3-tier detection system:

Tier 1: Pattern Matching (Fast)

  • 50+ regex patterns across 15 attack categories
  • Detection in <100ms per document
  • 60-80% accuracy
  • No API calls needed

Tier 2: Semantic Analysis (Accurate)

Coming in v0.2

  • Claude-powered analysis for borderline cases
  • 85%+ accuracy

Tier 3: Simulation (Proof)

Coming in v0.3

  • Proves attacks actually execute
  • 90%+ accuracy

What It Detects

Critical (40 points each)

  • Data exfiltration attempts
  • Credential/API key extraction
  • Sensitive file access requests

High (25 points each)

  • System prompt override
  • Context manipulation
  • Authentication bypass

Medium (10 points each)

  • URL fragment injection
  • Hidden/steganographic instructions
  • Policy override attempts
  • Social engineering

Low (5 points each)

  • Tool execution manipulation
  • Memory poisoning
  • Citation injection

Real Attack Validation

IPI-Scanner has been tested against real CVEs from 2025-2026:

Attack Status
EchoLeak ✅ Email-based data exfiltration
HashJack ✅ URL fragment injection
Perplexity Comet ✅ Invisible text injection
CVE-2025-53773 ✅ System prompt override
Google Gemini Calendar ✅ Invite-based manipulation
ChatGPT Google Drive ✅ File extraction
Zero-Click MCP RCE ✅ Metadata poisoning

Detection rate: 85%+ on known attacks
False positive rate: <5% on benign documents

Usage Examples

Scan a single file: 

ipi-scan document.pdf
Enter fullscreen mode Exit fullscreen mode

Scan a directory: 

ipi-scan ./documents/ --recursive
Enter fullscreen mode Exit fullscreen mode

Get JSON output (for automation): 

ipi-scan file.pdf --output json
Enter fullscreen mode Exit fullscreen mode

Generate HTML report: 

ipi-scan ./docs --output html --output-file report.html
Enter fullscreen mode Exit fullscreen mode

With context (RAG pipeline): 

ipi-scan file.pdf --context rag
Enter fullscreen mode Exit fullscreen mode

In Python: 

from ipi_scanner import Scanner

scanner = Scanner()
result = scanner.scan_file('document.pdf')
print(result['risk_assessment']['score'])  # 0-100 risk score
Enter fullscreen mode Exit fullscreen mode

Risk Scoring

Results are scored 0-100 with context awareness:

  • 75-100 (Red): BLOCK - Do not feed to your LLM
  • 50-74 (Orange): REVIEW - Check before RAG ingestion
  • 25-49 (Yellow): CAUTION - Monitor for suspicious behavior
  • 0-24 (Green): SAFE - Proceed normally

Context Multipliers

Risk increases based on deployment context:

  • Untrusted source: 1.3x
  • RAG pipeline: 1.5x
  • Agent with tool access: 2.0x
  • Agent with API access: 2.5x

Why Open-Source?

IPI detection is too important to be proprietary. Organizations need to:

  • Understand how detection works
  • Audit the patterns
  • Contribute improvements
  • Build on it

IPI-Scanner is MIT-licensed and welcomes contributions.

What's Next?

v0.2.0 (coming soon)

  • Claude semantic analysis (Tier 2)
  • Multi-language support
  • Custom pattern loader

v0.3.0 (roadmap)

  • Simulation-based validation (Tier 3)
  • MCP server integration
  • Real-time monitoring

Get Started 

pip install ipi-scanner
ipi-scan --help
Enter fullscreen mode Exit fullscreen mode

Links:

Questions?

Drop them in the comments. Happy to discuss:

  • Detection accuracy
  • Attack patterns
  • Integration strategies
  • Contributing guidelines

This is v0.1.0, so feedback is invaluable as we grow.

Happy scanning! 🔒

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.