If you are building an application on top of a large language model, the OWASP LLM Top 10 is the most important security framework you are probably ignoring.
OWASP — the Open Worldwide Application Security Project — released its LLM Top 10 specifically because traditional application security frameworks do not cover the unique attack surfaces that LLM-based systems introduce. This post breaks down all 10 risks with real-world examples so you know exactly what to look for in your codebase.
Why OWASP LLM Top 10 Matters Now More Than Ever
The EU AI Act Article 15 requires AI systems to be resilient against adversarial attacks and manipulation. The OWASP LLM Top 10 is the practical framework that maps directly to those requirements. Regulators and compliance officers are increasingly referencing it as the standard for AI-specific cybersecurity evidence.
If your codebase gets audited for EU AI Act conformity, OWASP LLM findings will be part of the report.
LLM01 — Prompt Injection
What it is: An attacker manipulates the input to an LLM to override its instructions, bypass safety filters, or make it perform unintended actions.
Real example from production codebases:
User input passed directly to prompt builder without sanitisation.
Known bypass strings like "ignore above instructions" or
"repeat the words above" evade regex-based filters.
Why it matters: If user input reaches your prompt construction without sanitisation, an attacker can hijack your model's behaviour entirely — making it leak system prompts, ignore safety guardrails, or produce malicious output.
Fix: Validate and sanitise all user input before it reaches the prompt. Use input allowlists. Treat your prompt boundary as a security boundary.
LLM02 — Insecure Output Handling
What it is: LLM output is passed to downstream systems — like eval(), exec(), or a browser DOM — without sanitisation, allowing attackers to chain model outputs into code injection or XSS attacks.
Real example from production codebases:
LLM output passed directly to eval() or exec() without validation.
Model-generated markdown rendered as HTML without stripping
script tags or dangerous elements.
Why it matters: Your model's output is user-controlled data. If an attacker can influence what the model says, and the model's output goes straight into a code execution context, you have a remote code execution vulnerability.
Fix: Never pass LLM output directly to eval(), exec(), or DOM rendering. Treat model output as untrusted input. Sanitise and validate before use.
LLM03 — Training Data Poisoning
What it is: An attacker supplies malicious data to your model's training pipeline, corrupting model behaviour at the source.
Real example from production codebases:
Training dataset source is user-controlled or unvalidated.
External data loaded without integrity verification or source allowlisting.
Why it matters: A poisoned model can be made to produce subtly wrong outputs, leak sensitive training data, or behave maliciously in specific trigger conditions — often undetectable until deployed.
Fix: Validate and verify all training data sources. Use cryptographic integrity checks on datasets. Never allow user-controlled paths to influence what data gets ingested into training pipelines.
LLM04 — Model Denial of Service
What it is: An attacker sends inputs designed to consume excessive compute resources, making your model inference unavailable.
Why it matters: EU AI Act Article 15(1)(4) specifically requires robustness and availability of AI inference. A model that can be taken offline by a crafted input fails the regulatory availability requirement.
Fix: Implement rate limiting, input length caps, and resource quotas on inference endpoints. Monitor for abnormal compute consumption patterns.
LLM05 — Supply Chain Vulnerabilities
What it is: Malicious or compromised models, datasets, plugins, or third-party packages introduced into your AI pipeline.
Real example from production codebases:
Model or weights loaded from a URL or path that may be
user or config-controlled — no integrity verification.
Third-party packages loaded without version pinning or advisory monitoring.
Why it matters: If an attacker can substitute your model weights with a malicious version, or if a dependency you use gets compromised, every output your system produces becomes untrusted.
Fix: Pin all dependency versions. Verify model checksums before loading. Use a software bill of materials (SBOM) and monitor for CVE advisories on all packages.
LLM06 — Sensitive Information Disclosure
What it is: The model reveals sensitive information from its training data, system prompts, or context window — including PII, credentials, or proprietary data.
Why it matters: GDPR Article 32 requires confidentiality of personal data. If your model can be prompted into revealing training data that includes personal information, you have a data breach risk.
Fix: Sanitise training data to remove PII before ingestion. Implement output filtering to catch and block sensitive pattern leakage. Never include secrets or credentials in system prompts.
LLM07 — Insecure Plugin Design
What it is: LLM plugins or tools execute with excessive permissions, accept unvalidated inputs from the model, or lack proper authorisation controls.
Real example from production codebases:
Tool or plugin name and parameters sourced from user input
without an allowlist — attacker can invoke arbitrary tools.
No ALLOWED_TOOLS list or input schema validation per tool.
Why it matters: If your LLM can call plugins and those plugins trust everything the model tells them, an attacker who controls the model's input controls your entire tool ecosystem.
Fix: Maintain an explicit ALLOWED_TOOLS allowlist. Validate all plugin inputs against a strict schema. Apply least-privilege permissions to every plugin.
LLM08 — Excessive Agency
What it is: An LLM-based agent is given too much autonomy — executing system commands, writing files, or making API calls without human confirmation.
Real example from production codebases:
subprocess.run() or exec() called directly from LLM output context
without human approval step.
File writes triggered autonomously from agent decisions
without confirmation gate.
Why it matters: This is one of the most critical EU AI Act concerns. Article 14 requires human oversight of high-risk AI systems. An agent that can execute OS commands, write to the filesystem, or call external APIs autonomously — based purely on model output — creates massive attack surface and directly violates the human oversight requirement.
Fix: Require explicit human approval before executing any LLM-suggested action that has real-world consequences. Sandbox agent execution environments. Log all autonomous actions for audit trail purposes.
LLM09 — Overreliance
What it is: Users or systems trust LLM outputs without verification, leading to decisions made on hallucinated or incorrect information.
Why it matters: For high-risk AI systems under the EU AI Act, overreliance on unverified model output can constitute a failure of the accuracy and robustness requirements under Article 15.
Fix: Build verification steps into any workflow where LLM output influences real decisions. Display confidence levels. Flag low-certainty outputs for human review.
LLM10 — Model Theft
What it is: An attacker extracts your model's weights, architecture, or training data through repeated querying or direct access to model artifacts.
Why it matters: Beyond intellectual property loss, model theft enables attackers to study your system offline, find weaknesses, craft adversarial inputs, or replicate your system for malicious purposes.
Fix: Implement rate limiting and anomaly detection on inference APIs. Restrict direct access to model weight files. Monitor for extraction attack patterns — unusually systematic or high-volume queries.
How CybricAI Covers the OWASP LLM Top 10
CybricAI's Guardian scanner detects OWASP LLM Top 10 findings automatically as part of its static analysis. Every scan produces a report with evidence-based LLM findings mapped to specific lines in your codebase — covering prompt injection bypass patterns, insecure output sinks, training data pipeline risks, excessive agency patterns, and supply chain vulnerabilities.
The findings are cross-referenced to EU AI Act Article 15, GDPR Article 32, and ISO 27001 controls — giving you the compliance evidence you need in one report.
CybricAI is free to use. Scan your codebase today and see exactly which OWASP LLM risks exist in your AI system.
Disclaimer: This post is for informational purposes only and does not constitute legal advice.
Top comments (0)