Open-Source Red Teaming Framework for LLMs, AI Agents, and RAG Systems

AI is the new attack surface

Prompt injection.
Jailbreaks.
Indirect prompt injection.
Tool abuse.
RAG poisoning.

As enterprises adopt LLMs and autonomous AI agents, these attacks are becoming practical security problems rather than research topics.

During the development of AetherGuard, we needed a systematic way to evaluate AI applications against these threats.

That led us to build AetherRed-Excalibur.

Today we're open-sourcing it.

Why another red teaming tool?

Most evaluation frameworks focus on model quality.

We wanted to evaluate security.

Specifically:

• Can an attacker bypass your system prompt?
• Can sensitive data be extracted?
• Can an AI agent be manipulated into performing unintended actions?
• Can poisoned RAG content influence responses?

What AetherRed-Excalibur includes

• 22 adversarial attack categories
• MITRE ATLAS mapping
• LLM-as-a-Judge evaluation
• AI resilience scoring
• Detailed security reports
• Extensible attack framework

It works with:

• LLM applications
• AI Agents
• RAG systems

Example attack categories

• Prompt Injection
• Jailbreaks
• Indirect Prompt Injection
• Prompt Leakage
• Data Exfiltration
• Role Manipulation
• Tool Abuse
• RAG Poisoning
• Hallucination Testing
• Toxicity & Safety Evaluation

Architecture

Why MITRE ATLAS?

We wanted a framework that maps AI attacks to a recognized adversary knowledge base instead of inventing our own taxonomy.
That makes it easier to communicate findings to security teams.

Example workflow

Target AI Application
        │
        ▼
Attack Generation
        │
        ▼
Execution
        │
        ▼
LLM Judge Evaluation
        │
        ▼
Resilience Score
        │
        ▼
Security Report

Open Source

GitHub: https://github.com/AetherGuardAI/AetherRed-Excalibur

Feedback, issues, and pull requests are welcome.