Ever wondered what a GPU goes through during a massive language model inference run? While you type a query and wait for tokens, the silicon under the hood is holding together a fragile house of cards: balancing context window limits, scheduling activations, managing weights, and evading malicious adversarial attacks.
To teach you how LLMs behave (and fall apart) under load, I built an interactive game:
⚡ GPU Survivors: Latent Space Hell
Play in Fullscreen Mode (if the embed sizing is tight)
🛠️ Choose Your Hardware Preset
Before initiating your run, choose your difficulty configuration (each represented by a unique retro pixel chip sprite and custom parameters):
- 🏢 Enterprise API (Easy): Spawns with 6 Core Integrity Lives, fast speed (
2.8), boosted damage, and a wide collection window. You get+25%XP gains and start with both the Attention Beam and the Softmax Aura active. - 💻 Consumer GPU (Medium): Spawns with 5 Core Lives, normal speed (
2.5), standard damage, and standard100%XP gains. Starts with the Attention Beam active. - 🍞 Smart Toaster (Hard): Edge inference on a kitchen appliance. Spawns with only 4 Core Lives, slow speed (
2.1), reduced damage, and a-20%XP penalty. Starts with a single Attention head active.
🧬 Playable ML Concepts Explained
This isn't just a homage to Vampire Survivors—every upgrade, weapon, and enemy represents a real-world concept in modern machine learning. Here is how the in-game mechanics map directly to how Large Language Models operate, fail, and optimize in production:
1. 🎯 Cosine Similarity (Piercing Vector Arrows)
- In-Game: Fires piercing vector arrows in a fan. Moving in the direction of the fire boosts damage by +60% (aligned vectors), while moving backward deals standard damage.
- The Real-World Counterpart: Text token embeddings are high-dimensional vectors. Cosine similarity calculates the cosine of the angle between two vectors to determine their semantic closeness: $$\text{Similarity} = \frac{\mathbf{A} \cdot \mathbf{B}}{|\mathbf{A}| |\mathbf{B}|}$$
- How it affects LLMs: This is the mathematical core behind Retrieval-Augmented Generation (RAG), semantic database search, and Self-Attention mechanisms. When the user prompt query vector aligns closely with a key vector in the model, the dot product spikes—assigning a massive attention score to pull that context forward.
2. 🗜️ Quantization (Passive Cooldown Upgrade)
- In-Game: Increases weapon firing rate (cooldown speed) at the cost of slightly lower base damage.
- The Real-World Counterpart: Quantization converts model weights and activation outputs from high-precision floating-point formats (like FP32 or FP16) to lower-precision integers (like INT8 or INT4).
- How it affects LLMs: Scaling massive models requires optimization. Quantization drastically reduces VRAM requirements, allowing a 70B parameter model that normally requires enterprise GPUs to run on local laptops. However, rounding values to a coarser scale introduces quantization noise, which slightly degrades the model's perplexity (leading to minor performance loss or "damage").
3. 🧬 Weight Decay (Hitbox Shrinking)
- In-Game: L2 regularization reduces the physical size of the player's core hitbox, making it harder for incoming token anomalies to land a hit.
- The Real-World Counterpart: L2 regularization penalizes large weights by adding a fraction of the squared magnitudes to the training loss function: $$L_{\text{regularized}} = L_{\text{base}} + \lambda \sum_{i} w_i^2$$
- How it affects LLMs: During pre-training, weight decay restrains model weights from growing too large. Keeping weights bounded makes the model less sensitive to minor noise in the user prompt, improving generalization and reducing hallucinations. The resulting "smaller footprint of instability" translates directly in-game to a smaller, more regularized core hitbox.
4. 🧬 Node Dropout (Ignore Hit Check)
- In-Game: Grants a flat +8% chance per level to completely ignore or evade incoming damage.
- The Real-World Counterpart: Dropout is a regularization technique where a random percentage of neural nodes (activations) are zeroed out at each training step.
- How it affects LLMs: By shutting down random neural pathways during training, the model is forced to learn redundant, robust representations rather than relying on a single, fragile sequence of nodes. This prevents the model from overfitting to its training dataset, allowing it to adapt cleanly to unseen prompt distributions at inference time (represented in-game by dropping nodes to safely "evade" bad data).
5. 🔒 Adversarial Split (Jailbreaks)
- In-Game: High-health golden locks. When destroyed, they split into 3 fast-moving Adversarial Tokens that lock onto the player.
- The Real-World Counterpart: A jailbreak is a targeted input sequence designed to bypass the safety alignments (RLHF/DPO) of a model, prompting it to output restricted content.
- How it affects LLMs: Jailbreaks exploit the fact that LLMs treat data and instructions identically. Once a malicious prompt slips past the model's safety guardrails, it triggers an autoregressive cascade of toxic outputs. In-game, this is represented by the sudden explosion of fast-moving adversarial tokens that quickly clutter your active context window.
6. ⚖️ The Horizontal Data Bias (Skewing Fields)
- In-Game: Stepping inside the green Data Bias radius skews your movement coordinate vectors, dragging you in the direction the arrow points.
- The Real-World Counterpart: Data bias occurs when training corpora contain unbalanced representations, stereotypes, or uneven historical distributions.
- How it affects LLMs: LLMs reflect their training datasets. If the data is biased, the output token probability distribution is heavily skewed toward those prejudices. For example, if a model's training data repeatedly associates a profession with a specific demographic, it will struggle to generate neutral completions. This creates a constant, invisible drift that biases output completions, directly mirroring the in-game dragging force.
7. 💾 KV-Cache (The Protective Orbitals)
- In-Game: Key-Value caching blocks rotate around the core, absorbing hits.
- The Real-World Counterpart: The KV-Cache saves the key-value representations of past tokens in GPU VRAM so they don't have to be recalculated at every token prediction step.
- How it affects LLMs: Auto-regressive generation predicts one word at a time, feeding its own output back as input. Without a KV-cache, the model would have to compute attention scores across the entire history for every single token generated, causing latency to scale quadratically. The KV-cache saves computation but consumes huge amounts of memory, restricting user concurrency.
💀 The 15-Minute Thermal Runaway (Endgame)
At exactly 15:00, all standard enemies are swept away, and the unkillable red boss Hardware Degradation arrives. You cannot harm it.
💬 Let's Discuss:
- What was your longest survival time on Smart Toaster difficulty?
- What architectural combination (e.g., Quantization speed boosts + Cosine Similarity) did you find most effective?
UnitBuilds-CC
/
GPU-SURVIVORS
Homage to Vampire Survivors, as an educational game to teach players about LLMs.
GPU Survivors: Latent Space Hell 📟⚡
Can you survive a 1T parameter inference run?
Welcome to GPU Survivors, an interactive 2D retro action-roguelike built to simulate the architectural limits, failure modes, and optimization hyperparameters of running a Large Language Model under load.
🎮 The Scenario
In the digital deep, bad data and chaotic vectors threaten inference stability. You are a GPU Core initializing a new language model. Survive the endless incoming waves of training loads (OOD outliers, prompt injections, and data biases), gather FLOPs (XP), and scale your architecture to 1T parameters!
⌨️ Controls
-
Move: Use
WASDorArrow Keys. -
Pause: Press
EscapeorPto pause the run, resume, or exit. - Attack: Auto-targeted active weapons fire queries at the nearest token anomalies.
🛠️ Hardware Presets (Difficulty Modes)
Select your inference endpoint difficulty at startup:
-
🏢 ENTERPRISE API (H100 Cluster) — Easy
- Stats: 6 Core Integrity…
Disclaimer: AI was used throughout this project, it is just fitting that it would co-author with me, so special thanks to the Foundry for its tireless hours toiling away and Gemini for producing the cover image.
Top comments (1)
Please @ someone who can review the PR to fix the Embed window sizing, so the games actually look and run nicely inside a post: github.com/forem/forem/pull/23553
Thank you and enjoy the game!