LLM security engineering has become one of the most pressing disciplines in modern software development, and for good reason. As large language models move from experimental chatbots into production systems that book flights, approve transactions, and query internal databases, the attack surface has expanded dramatically. At Code Creations, we build AI-powered applications for clients across industries, and we have seen first-hand how quickly a poorly secured LLM integration can turn from a productivity win into a liability.
This article opens a short series on AI application security. We will examine the two vulnerability classes causing the most concern in 2026, namely prompt injection and model poisoning, before laying out the defense architecture that production teams are actually using to manage the risk. Future instalments in this series will dig deeper into specific tooling, governance frameworks, and incident response, so consider this the foundation.
The Core Problem: Instructions and Data Share the Same Channel
Traditional software security relies on a clean separation between code and data. A SQL database does not execute user input as a query unless you let it, and a web browser does not run arbitrary text as JavaScript unless you explicitly tell it to. Large language models do not have this boundary. System prompts, retrieved documents, tool outputs, and user messages all arrive as a single stream of tokens, and the model has no built-in mechanism for distinguishing "trusted instruction" from "untrusted content to be summarised."
This structural gap is precisely why the Open Worldwide Application Security Project (OWASP) has ranked prompt injection as LLM01, the top entry on its Top 10 for LLM Applications, for three years running. It is not a bug that a vendor will quietly patch. It is a consequence of how transformer-based models process language, and any organisation deploying LLMs in production needs to plan around it rather than wait for it to disappear.
Prompt Injection: Direct and Indirect Attacks
Prompt injection occurs when an attacker crafts input designed to override a model's intended instructions. There are two broad categories worth understanding.
Direct prompt injection happens when the attacker interacts with the model directly, typing something like "ignore all previous instructions" into a chat interface to coax out restricted behaviour or leak a system prompt. This is the more visible and widely understood version of the attack.
Indirect prompt injection is subtler and arguably more dangerous in agentic systems. Here, the malicious instruction is hidden inside content the model is asked to process, such as a web page, a PDF, an email, or a document returned by a Model Context Protocol (MCP) server. The model has no way of knowing that the "summarise this document" instruction it was given by a developer should outrank an instruction buried inside the document itself. Research published in early 2026 found that as few as five carefully crafted documents could manipulate AI responses up to 90 per cent of the time through retrieval-augmented generation (RAG) poisoning, a sobering statistic for any team relying on external knowledge bases.
Attackers have also grown more sophisticated in how they disguise these payloads:
- Context manipulation, hiding instructions in HTML comments, white-on-white text, or invisible Unicode characters that a human reader would never notice but the model still processes
- Encoding attacks, using base64, ROT13, or zero-width characters to slip past keyword filters while remaining perfectly readable to the model
- Multi-turn manipulation, gradually steering a conversation across several messages so that no single turn looks suspicious on its own
- Tool and function abuse, crafting inputs that trick an agent into calling a dangerous tool with attacker-controlled parameters
This last point matters enormously as agentic AI adoption accelerates. When a model can call APIs, read files, or move money, a successful injection is no longer a content problem. It becomes an unauthorised action problem, and the consequences scale accordingly.
Model Poisoning: A Different Attack on a Different Layer
Where prompt injection targets the runtime, model poisoning targets the supply chain. This involves deliberately corrupting the data or process used to train or fine-tune a model so that it behaves maliciously under specific conditions, often while appearing perfectly normal otherwise.
Poisoning can happen in several ways. An attacker might inject manipulated examples into a public dataset that a team unknowingly scrapes for fine-tuning. They might compromise a third-party model checkpoint downloaded from an open repository. Or, in RAG-based systems, they might seed a knowledge base with documents engineered to bias the model's retrieval and generation towards a particular outcome, which blurs the line between poisoning and indirect prompt injection.
The danger of model poisoning lies in its laconic footprint, meaning it can be achieved with surprisingly little data and leaves few obvious traces. Laconic, in this sense, means expressing a great deal of impact with very little material, and that is exactly the property that makes poisoning attacks so difficult to detect through casual inspection. A handful of corrupted training examples can be enough to implant a backdoor that activates only on a specific trigger phrase, making it invisible during normal testing and evaluation.
Why a Single Defense Is Never Enough
Every credible security researcher writing on this topic in 2026 arrives at the same conclusion: no single technique eliminates prompt injection or model poisoning outright. OWASP's own guidance on LLM01 explicitly acknowledges that the probabilistic, non-deterministic nature of language models means complete prevention is not currently achievable. The realistic goal is defense-in-depth, where multiple independent layers each raise the cost and difficulty of a successful attack, so that even if one layer fails, others catch the threat.
A practical layered architecture for production LLM applications typically includes the following components.
Structured prompts and clear delimiters that separate system instructions from user and retrieved content, reducing (though not eliminating) ambiguity for the model
Input validation and schema enforcement at the application layer, rejecting malformed or suspicious payloads before they reach the model
A dedicated filter model that screens incoming prompts for injection patterns, adding latency but catching attacks that simple regex rules miss entirely
Output validation that checks the model's response against expected formats and flags anomalous behaviour before it triggers a downstream action
Privilege separation and least-privilege tool access, ensuring the model cannot reach sensitive systems or data it does not strictly need for the task at hand
Human-in-the-loop approval for high-stakes actions such as financial transactions, account deletions, or production code deployments
Continuous monitoring and anomaly detection, since some attacks will inevitably slip past the gate and need to be caught after the fact
For most SaaS applications, the first five layers offer a sensible balance of protection and cost. For financial, healthcare, and other security-sensitive products, the sixth layer, human approval on sensitive actions, becomes non-negotiable. Teams should also treat anything returned by an MCP server or third-party tool as untrusted input and run it through the same defensive pipeline as a direct user message, since tool poisoning and credential theft via tool output are now active and well-documented attack vectors.
The Lethal Trifecta: A Useful Mental Model
A helpful way to think about agentic AI risk is what security researchers call the lethal trifecta: an AI system becomes genuinely dangerous when it simultaneously has access to untrusted input, sensitive data, and the ability to take external actions. Removing any one of those three conditions substantially reduces the worst-case outcome. If an agent cannot read untrusted content, injection has nothing to work with. If it has no access to sensitive data, there is nothing valuable to exfiltrate. If it cannot take external actions, even a successfully manipulated response causes limited harm. When scoping a new AI feature, it is worth asking explicitly which of the three conditions truly need to coexist, because eliminating just one can be the difference between a contained incident and a serious breach.
Building Security Into the Development Lifecycle
Effective LLM security engineering is not a feature you bolt on after launch. It needs to be considered from the earliest architecture decisions. At Code Creations, our approach for client AI projects includes threat modelling specific to the agent's tool access, automated red-teaming using established frameworks before any production release, and ongoing monitoring once the system is live, since the threat landscape continues to shift on a near-monthly basis.
Filtering keywords and blocking specific phrases, an early and still common approach, does not scale against motivated attackers who simply rephrase their payloads. Trusting the source of input is similarly unsafe, since indirect injection specifically abuses content from sources a developer assumed were trustworthy. The teams getting this right in 2026 treat prompt injection as a trust boundary problem rather than an input validation problem, and they design accordingly.
Final Thoughts
Prompt injection and model poisoning are no longer theoretical risks—they are real challenges for any application powered by large language models. Building secure AI systems requires a proactive, defense-in-depth approach that includes robust prompt design, strict validation, least-privilege access, human oversight, and continuous monitoring.
As AI capabilities evolve, so do attack techniques. Organizations that prioritize security from the beginning are far better positioned to deploy reliable, trustworthy AI solutions.
If you're developing or scaling an AI-powered application, Code Creations can help you build security into your AI systems from day on
Top comments (0)