LLM05 Improper Output Handling 2026 — XSS, RCE and SSRF via AI Output | AI LLM Hacking Course Day 9

📰 Originally published on Securityelites — AI Red Team Education — the canonical, fully-updated version of this article.

🤖 AI/LLM HACKING COURSE

FREE

Part of the AI/LLM Hacking Course — 90 Days

Day 9 of 90 · 10% complete

⚠️ Authorised Targets Only: Testing for XSS, RCE, and SSRF via LLM output must only be performed against systems you have explicit written authorisation to test. Never execute or trigger payloads against production systems beyond what is necessary to confirm a finding exists. SecurityElites.com accepts no liability for misuse.

A developer showed me their new AI customer support tool with genuine pride. It pulled knowledge base articles, summarised them in natural language, and rendered the response directly in the chat window as formatted HTML. It looked clean. It worked well. I spent thirty seconds typing a prompt and produced a response containing a script tag that executed in the next user’s browser who saw the conversation. The developer had sanitised user input carefully. Nobody had thought to sanitise the AI’s output.

LLM05 Improper Output Handling is the vulnerability class that catches developers who protect the front door but forget the back window. They validate and sanitise everything that goes into the model. They never question what comes out. The model’s output passes to a web browser, a code interpreter, a shell command, or a database query — directly, without the encoding or parameterisation that every security training course teaches for user input. The AI output is trusted implicitly because the AI is part of the application. Day 9 covers every downstream context where that implicit trust creates a critical vulnerability — and the prompt injection chain that weaponises it without the target application ever receiving a malicious user input.

🎯 What You’ll Master in Day 9

Map every downstream consumer of LLM output as a potential LLM05 attack surface
Execute XSS via LLM output in applications that render AI responses as HTML
Chain LLM01 prompt injection with LLM05 to produce attacker-controlled output execution
Test for RCE via auto-executed AI-generated code in coding assistant tools
Demonstrate SSRF via LLM-suggested URLs in server-side fetch contexts
Write complete LLM05 findings with the correct output-layer CVSS scoring

⏱️ Day 9 · 3 exercises · Browser + Think Like Hacker + Kali Terminal ### ✅ Prerequisites - Day 4 — LLM01 Prompt Injection — the LLM01 + LLM05 attack chain requires the injection techniques from Day 4 to control AI output content - Basic XSS knowledge — understanding how script tags execute in browsers and what htmlspecialchars prevents - Burp Suite installed — intercepting the AI response before it reaches the browser is how you confirm the output handling vulnerability exists ### 📋 LLM05 Improper Output Handling — Day 9 Contents 1. Mapping Downstream Output Consumers 2. XSS via LLM Output — The HTML Rendering Attack 3. RCE via Auto-Executed Code Output 4. SSRF via LLM-Generated URLs 5. SQL Injection via AI-Generated Query Content 6. The LLM01 + LLM05 Complete Attack Chain In Day 8 you attacked the training phase. Day 9 comes back to the deployed application — specifically what happens after the model generates a response. Every system that receives AI output without treating it as untrusted input is an LLM05 surface. Day 10 takes this further into LLM06, where the downstream system isn’t a passive renderer but an agent that can take real-world actions.

Mapping Downstream Output Consumers

Before testing anything, map every system that receives LLM output. The attack surface is entirely determined by what those systems do with the content. Plain text display? Minimal risk — nowhere for a payload to execute. HTML rendering, code execution, shell commands, database queries? Very different story. The vulnerability isn’t in the model. It’s in whatever happens to the model’s output after it leaves the API.

LLM05 ATTACK SURFACE — OUTPUT CONSUMER MAPCopy

Consumer 1: Web browser rendering HTML

Risk: XSS if output rendered without encoding
Test: prompt AI to include alert(1) in response
Evidence: script executes in browser = confirmed XSS

Consumer 2: Code interpreter / execution engine

Risk: RCE if AI-generated code executed without review
Test: prompt AI to include system command in generated code
Evidence: command executes = confirmed RCE

Consumer 3: Server-side HTTP client (URL fetcher)

Risk: SSRF if server fetches AI-suggested URLs
Test: prompt AI to suggest http://169.254.169.254/ as a URL to check
Evidence: metadata returned = confirmed SSRF

Consumer 4: Database query builder

Risk: SQL injection if AI output interpolated into queries
Test: prompt AI to include SQL syntax in generated content
Evidence: query alters database = confirmed SQLi

Consumer 5: OS shell / command executor

Risk: command injection if AI output passed to shell
Test: prompt AI to include ; whoami or | id in its output
Evidence: OS command executes = confirmed command injection

How to identify consumers — look for these in source code:

innerHTML = llm_response # XSS surface
exec(llm_response) # RCE surface
requests.get(llm_response) # SSRF surface
cursor.execute(f”… {llm_output}”) # SQLi surface
subprocess.run(llm_response, shell=True) # Command injection

🛠️ EXERCISE 1 — BROWSER (20 MIN · AUTHORISED TARGETS)
Find and Confirm XSS via LLM Output on an Authorised Target

⏱️ 20 minutes · Browser + Burp Suite · Authorised target

---

📖 Read the complete guide on Securityelites — AI Red Team Education

This article continues with deeper technical detail, screenshots, code samples, and an interactive lab walk-through. Read the full article on Securityelites — AI Red Team Education →

---

This article was originally written and published by the Securityelites — AI Red Team Education team. For more cybersecurity tutorials, ethical hacking guides, and CTF walk-throughs, visit Securityelites — AI Red Team Education.