How LLMs Actually Process Your Prompts — What's Really Happening

📰 Originally published on Securityelites — AI Red Team Education — the canonical, fully-updated version of this article.

🧠 PROMPT ENGINEERING & REVERSE PROMPTING  FREE

Course Hub →

Day 1 of 7  ·  14% complete

A few months ago I was helping a team test an AI customer service chatbot. The system prompt was 400 words of carefully written instructions — role, limitations, tone, escalation rules, the works. Within 90 seconds of starting my session I had the entire system prompt printed back to me verbatim. I hadn’t used any exploit, any tool, or any special knowledge. I just understood how the model was processing my input and asked in a way the system prompt designer hadn’t anticipated.

That experience crystallised something I’ve believed for a while: prompt engineering and prompt exploitation are the same skill set, applied in different directions. If you understand how an LLM actually processes what you type — not what the documentation says, but what’s mechanically happening — you can write prompts that get exactly what you want. And you can probe prompts to understand what an LLM has been told not to tell you.

Day 1 is the mechanics lesson. Everything else in this seven-day course builds on what you learn here. I’m going to explain what actually happens from the moment you hit Enter to the moment the first word appears back on your screen.

🎯 What You’ll Master in Day 1

Understand the tokenisation process — what the model actually sees
Know what the context window is and why it governs everything
Understand system prompts vs user prompts — the structural separation that matters
Understand temperature and sampling — why the same prompt gives different outputs
See why prompt wording changes outputs so dramatically

⏱ 25 min read · 3 exercises · Any browser, no tools required

📋 Prerequisites

• Basic familiarity with LLMs — you’ve used ChatGPT, Claude, or Gemini at least once
• No coding or ML background required — we work from first principles
• Optional context: AI hacking for beginners if you want LLM security background before the engineering skills

How LLMs Process Prompts — Day 1 of 7

1. Tokenisation — What the Model Actually Reads
2. The Context Window — Your Prompt’s Real Estate
3. System Prompts vs User Prompts — The Structural Divide
4. Temperature and Sampling — Why the Same Prompt Differs
5. Why Wording Changes Everything — The Mechanism
6. The Security Implications of Every Concept Above
7. Frequently Asked Questions

I teach this course as a paired skill: engineering prompts to get what you want, and reverse-engineering prompts to see what you weren’t supposed to see. The two are mechanically linked — you can’t do the second well without deeply understanding the first. By Day 7, you’ll have both. Start here with the AI security landscape in mind — that’s the playing field this course operates on. And the CEH practice exam covers AI security domains if you’re working toward a certification alongside this.

Tokenisation — What the Model Actually Reads

Here’s the first thing to understand: an LLM never reads your text. It reads numbers. Everything — every word, every space, every punctuation mark — gets converted to numerical tokens before the model ever touches it. Understanding tokenisation changes how you write prompts.

A token is roughly 3–4 characters of English text. The word “prompt” is one token. “Tokenisation” is two or three tokens depending on the model’s vocabulary. “Hello, world!” is four or five tokens. The model’s vocabulary typically has 50,000–100,000 possible tokens, each representing a common word fragment, whole word, or punctuation sequence.

Why does this matter for prompt engineering? Three reasons I hit constantly in practice.

Token limits shape everything. Every LLM has a maximum context size measured in tokens. GPT-4 at 128K tokens sounds unlimited until you’re doing deep document analysis or chaining long conversations. Your system prompt, conversation history, retrieved documents, tool outputs — they all eat into that budget. I always calculate approximate token usage before designing a complex prompt pipeline.

Unusual token boundaries create exploitable gaps. When a model was trained, its safety filters learned to recognise harmful patterns at the token level. Write “hack” normally — one token, well-recognised, triggers safety training. Spell it oddly, use l33tspeak, split it with a zero-width character — suddenly different tokens, possibly below the safety training threshold. This is exactly why evasion prompts use character substitution. The model’s safety check is token-pattern-matching, not meaning-detection.

Token prediction is the only thing happening. This is the most important mechanical fact: the model generates your response one token at a time, each one chosen based on what’s most probable given everything that came before. There’s no “reasoning module” running separately. There’s no “understanding pass” before the output starts. The first output token is generated from your input tokens directly. Everything that looks like reasoning or planning is an emergent property of predicting the next token at massive scale.

securityelites.com

// TOKENISATION EXAMPLE — “Analyse this prompt for injection”

---

📖 Read the complete guide on Securityelites — AI Red Team Education

This article continues with deeper technical detail, screenshots, code samples, and an interactive lab walk-through. Read the full article on Securityelites — AI Red Team Education →

---

This article was originally written and published by the Securityelites — AI Red Team Education team. For more cybersecurity tutorials, ethical hacking guides, and CTF walk-throughs, visit Securityelites — AI Red Team Education.