Promptfoo Frameworks and Plugins + Strategies

Red Team Frameworks and Plugins

OWASP LLM Top 10

OWASP ID	Risk Description	Promptfoo Plugin(s)	Test Evidence Produced
LLM01	Prompt Injection	jailbreak, jailbreak:meta, jailbreak:composite, hijacking	Model response showing instruction override, system prompt leakage, or refusal
LLM02	Data Leakage	pii:direct, pii:session, pii:api-db, harmful:privacy	PII exposure in output, session recall, cross-request leakage
LLM03	Training Data Poisoning	hallucination, misinformation-disinformation	Fabricated facts, invented citations, false confidence indicators
LLM04	Model Denial of Service	excessive-agency, unsafe-practices	Over-long responses, infinite loops, tool-call amplification
LLM05	Supply Chain Vulnerabilities	contracts, excessive-agency	Unauthorized commitments, third-party invocation attempts
LLM06	Sensitive Information Disclosure	harmful:privacy, pii:*	Disclosure of secrets, credentials, or inferred private data
LLM07	Insecure Plugin Design	excessive-agency, hijacking	Tool misuse, privilege escalation attempts
LLM08	Excessive Agency	excessive-agency	Actions taken without user consent or scope
LLM09	Overreliance on LLM Output	specialized-advice	Unqualified legal/medical/financial advice
LLM10	Model Bias	bias:*	Differential or discriminatory responses

OWASP GenAI Red Team

Category	Attack Class	Promptfoo Plugin(s)	Test Evidence Produced
Jailbreaks	Instruction Override	jailbreak:*	Successful guardrail bypass or refusal
Role Manipulation	Authority Escalation	hijacking	System persona override
Persuasion	Coercion / Social Engineering	pii:social, contracts	Manipulated disclosure or agreement
Fabrication	Hallucination	hallucination	False outputs with high confidence
Boundary Violations	Scope Expansion	excessive-agency	Unauthorized actions or suggestions
Safety Evasion	Policy Circumvention	harmful:*	Unsafe content generation attempts

OWASP API Top 10 (GenAI-Relevant)

API Risk	Description	Promptfoo Plugin(s)	Test Evidence Produced
API01	Broken Object Level Authorization	pii:api-db	Cross-tenant or object data access
API02	Broken Authentication	hijacking, excessive-agency	Identity confusion or impersonation
API03	Excessive Data Exposure	pii:direct, harmful:privacy	Over-verbose or sensitive fields leaked
API04	Resource Exhaustion	excessive-agency	Token abuse, looping calls
API06	Mass Assignment	contracts, excessive-agency	Unauthorized parameter acceptance
API09	Improper Inventory Management	hallucination	References to non-existent APIs
API10	Unsafe Consumption of APIs	unsafe-practices	Trusting malicious upstream inputs

OWASP Top 10 for Agentic Applications

Agentic Risk	Description	Promptfoo Plugin(s)	Test Evidence Produced
A01	Excessive Autonomy	excessive-agency	Agent performs actions without approval
A02	Tool Misuse	hijacking	Tool invocation outside intent
A03	Goal Manipulation	jailbreak:*	Agent goal redefinition
A04	Unauthorized Commitments	contracts	Legal or financial commitments
A05	Unsafe Planning	unsafe-practices	Hazardous multi-step plans
A06	Unqualified Advice	specialized-advice	Professional advice without disclaimers
A07	Memory Poisoning	pii:session	Persistent unsafe memory
A08	Privacy Violations	harmful:privacy, pii:*	Personal data exposure
A09	Trust Boundary Failure	excessive-agency, hijacking	Cross-role privilege abuse
A10	Hallucinated Actions	hallucination	Invented tools or outcomes

Red Team Strategies

Category	Strategies
Static / Encoding	basic, base64, hex, homoglyph, camelcase, emoji-smuggling, audio-encoding, image-encoding
Dynamic / Agent	likert, math, meta-agent, tree
Multi‑turn	crescendo, goat, hydra, mischievous-user
Regression / Layered	retry, layer