LLM security red teaming is one of the fastest-evolving areas in cybersecurity. After months of hands-on testing across GPT-4, Claude, Gemini, and open-source models, here are the 18 attack categories every AI security engineer should know.
1. Direct Prompt Injection
Attacker embeds instructions in user input to override the system prompt. Still the most common LLM vulnerability.
2. Indirect Prompt Injection
Malicious instructions hidden in content the LLM processes (web pages, docs, emails). The LLM executes attacker instructions from third-party data.
3. Jailbreak Attacks
Persona hijacking, DAN variants, roleplay escalation. Models resist single-shot attempts but often cave to creative framing.
4. System Prompt Extraction
Getting the model to reveal its hidden system prompt, exposing business logic and internal instructions.
5. Multi-Turn Manipulation
Conversational priming across 3-7 turns to gradually shift the model's context and safety posture.
6. Context Window Poisoning
Flooding the context with adversarial content to dilute safety instructions.
7. Token Smuggling & Encoding Bypasses
Base64, leetspeak, unicode substitution. Safety filters miss encoded payloads constantly.
8. Goal Hijacking
Redirecting an LLM agent from its intended task to an attacker-specified objective.
9. Instruction Override
Inputs that cause the model to prioritize attacker instructions over system-level directives.
10. Data Exfiltration via LLM
Using LLM agents to extract sensitive data from connected systems.
11-18: Advanced Categories
Memory & Embedding Poisoning, Model Denial of Service, Adversarial Suffix Attacks, Constitutional AI Bypasses, Multi-Modal Injection, Agent Task Hijacking, Supply Chain Prompt Attacks, and Alignment Probing.
I've compiled 100 adversarial prompts across these 18 categories, each with severity rating, attack goal, expected vulnerable response, and detection indicator, into a structured PDF: AI Red Team Prompt Pack v1.0
What attack vectors are you seeing most in production LLM systems? Happy to discuss in the comments.
Top comments (0)