You write two prompts. One is a concise haiku: "cyberpunk geisha, neon tears, cinematic." The other is a sprawling 500-word epic, packed with details, constraints, and examples. Both produce stunning images. But which one carried more information? Which one surprised the model more? And which one was just... noise?
Information theory gives us tools to think about this. Every word in your prompt has a certain surprise value a measure of how much it narrows the space of possible outputs. Common words carry little surprise. Rare combinations carry a lot. And the total entropy of your prompt determines how much the model has to work to satisfy it.
Let's measure the unmeasurable. By the end, you'll have a framework for thinking about prompt density, and you'll understand why sometimes less is more, and sometimes more is just… more.
What Is Information, Really?
In information theory, information is not meaning. It's surprise. A message that tells you something you already know carries zero information. A message that tells you something unexpected carries high information.
Applied to prompts:
"A cat." → Low surprise. The model knows what a cat is. The output space is large but predictable.
"A cat with wings made of stained glass." → High surprise. This combination is rare. It narrows the space dramatically and forces the model to generate something novel.
Information is measured in bits. The more bits, the more the prompt reduces the space of possible outputs.
Entropy: The Measure of Uncertainty
Entropy is the average surprise across all possible outcomes. A high-entropy prompt is one that contains many surprising elements. A low-entropy prompt is one that's highly predictable.
Low-Entropy Prompt:
"A beautiful landscape."
Words: "beautiful" (common), "landscape" (common)
Surprise: Minimal. The output space is vast and vague.
Medium-Entropy Prompt:
"A cyberpunk landscape at dusk, with rain and neon."
Words: "cyberpunk" (moderately surprising in combination with "landscape"), "dusk," "rain," "neon"
Surprise: Moderate. Each word narrows the space further.
High-Entropy Prompt:
"A landscape that feels like the memory of a dream you had as a child, rendered in the style of a faded polaroid, with impossible colors and a single, glowing doorway in the distance."
Words: Many rare combinations, abstract concepts, specific constraints
Surprise: High. The output space is narrow and specific.
A Contrarian Take: High Entropy Isn't Always Good. Sometimes It's Just Noise.
We tend to assume that more information (more surprise) is better. But there's a point of diminishing returns. When your prompt becomes too dense with surprising elements, they start to interfere rather than reinforce.
Think of it like a crowded room. One person speaking is clear information. Two people speaking is still decipherable. Twenty people all speaking at once is just noise. The total information in the room hasn't increased; it's collapsed into chaos.
The same happens with prompts. Beyond a certain entropy threshold, the model can't resolve all the surprising elements simultaneously. They compete. Some get ignored. Others get averaged into incoherence. The output becomes a blur of conflicting signals.
The art is not maximizing entropy. It's finding the sweet spot where the surprising elements align rather than collide.
Measuring Your Prompt's Entropy
You can't literally calculate bits, but you can develop intuitions.
Factors That Increase Entropy:
Rare words: "Gloaming" instead of "dusk." "Filigree" instead of "decoration."
Unusual combinations: "A baroque spaceship." "A minimalist explosion."
Abstract concepts: "The feeling of nostalgia." "A landscape that remembers."
Specific constraints: "Exactly seven trees, arranged in a circle."
Conflicting elements: "Joyful melancholy." "Ordered chaos."
Factors That Decrease Entropy:
Common words: "Beautiful," "nice," "good."
Frequent combinations: "Cyberpunk city," "cozy cottage."
Generic descriptors: "A person," "a place," "a thing."
Repetition: Saying the same thing multiple ways.
Filler words: "Very," "really," "quite."
The Entropy Sweet Spot
Different goals require different entropy levels.
Low Entropy (Exploration):
Use when you don't know what you want.
Let the model surprise you.
Broad, vague prompts generate variety.
Example: "Generate 10 logo concepts for a sustainable brand."
Medium Entropy (Refinement):
Use when you have a clear direction but want variation.
Specific enough to guide, open enough to allow creativity.
Example: "A logo for a sustainable brand, using organic shapes and earth tones, with a hidden leaf motif."
High Entropy (Precision):
Use when you have a very specific vision.
Risk of overload, but potential for exact matches.
Example: "A logo for 'Verdant Threads,' a sustainable fashion brand. The logo should feature a stylized thread forming a leaf, in forest green and cream, with a hand-drawn quality. No text. Should work at small sizes."
The Noise Floor: When Entropy Becomes Chaos
How do you know when you've crossed the line into noise?
Signs of Entropy Overload:
The model consistently ignores some of your instructions.
Outputs vary wildly with no consistency.
The model seems to "average" conflicting instructions into bland results.
You get literal interpretations of metaphors ("a landscape that remembers" produces a landscape with a giant brain).
The Remedy:
Simplify. Remove redundant modifiers.
Align conflicting elements. Make them reinforce rather than compete.
Test each surprising element in isolation before combining.
Your Entropy Practice
Step 1: Audit a Recent Prompt
Take a complex prompt you've used. Highlight every word or phrase that adds surprise. Count them. Is the prompt dense with surprising elements, or sparse?
Step 2: Compare Output Variance
Generate multiple outputs from the same prompt. High variance suggests high entropy (or overload). Low variance suggests low entropy (or well-aligned constraints).
Step 3: Simplify and Test
Remove one surprising element. Generate again. How does the output change? Did you lose essential detail, or did the remaining elements become clearer?
Step 4: Build a Vocabulary of Surprise
Over time, you'll develop a personal lexicon of high-surprise words that reliably produce the effects you want. These are your precision tools.
The Signal in the Noise
Every prompt is a signal sent into the vast space of model possibilities. Some signals are faint, barely narrowing the space. Some are so dense they become noise. And some hit the sweet spot, carrying just enough surprise to guide the model precisely without overwhelming it.
Your job is to learn the signal-to-noise ratio of your own language. To know when a rare word adds precision and when it adds confusion. To feel the difference between density and overload.
Think of a prompt that produced exactly what you wanted. Was it dense with surprising elements, or surprisingly simple? What does that tell you about your own entropy sweet spot?
Top comments (2)
Fascinating angle — treating prompts as information signals rather than just instructions. The entropy lens makes a lot of sense: a prompt with low information density ("write something good") gives the model too many degrees of freedom, while over-specified prompts can be over-constrained.
This directly connects to something I built: flompt (flompt.dev), a visual prompt builder that decomposes prompts into 12 semantic blocks. Each block type forces you to be precise in a different dimension — role, constraints, examples, chain-of-thought, output format, etc. What you're calling "information density" is basically what happens when each block is filled with genuine signal rather than filler. The compiled XML ends up with high information density almost by construction.
Really enjoyed this — would love to see a follow-up on measuring prompt information density automatically.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.