My support agent ran on a 900-word instruction prompt that I'd been polishing for two months. Every time it made a mistake, I added another rule. "Never promise refunds above $50 without flagging." "If the customer is angry, acknowledge first." "When unsure about shipping times, say 5–7 business days, not 3–5."
Nine hundred words of rules, and the agent still occasionally sounded like a lawyer apologizing for existing.
So in July I tried the thing every prompt engineering article tells you to do: I deleted most of the instructions and replaced them with six hand-written examples of ideal responses. Few-shot prompting. Supposedly a slam dunk.
It got worse immediately — in a way I didn't understand for about a week. This post is what I learned: why examples beat instructions most of the time, the specific failure mode nobody warned me about, and the structure I ended up with.
Why examples beat instructions (when they work)
Instructions describe the target. Examples are the target. A model doesn't have to interpret "be concise but warm" — those words map to a fuzzy region of behavior space. But a concrete example of a concise-but-warm reply pins the tone, length, and format exactly, with zero interpretation overhead.
Three things improved the day I switched:
1. Format compliance went from ~85% to ~99%. I'd been begging the agent (in prose) to always end shipping questions with the tracking-link pattern. One example that ended that way did more than three paragraphs of rules.
2. Tone stopped drifting. With instructions, tone depended on which rules the model happened to weight on a given run. With examples, the tone was anchored. Every output sounded like it came from the same human.
3. The prompt got shorter and cheaper. Six examples plus 120 words of framing cost fewer tokens than my 900-word rulebook, and I could finally read the whole prompt in one screen.
If that were the whole story, this would be a boring post.
The failure: my agent started hallucinating my examples
Here's what nobody told me. On day three, a customer asked about a product I don't even sell anymore — an old plan called "Starter Tier" that I'd retired in 2024. One of my six examples happened to mention Starter Tier pricing, because I'd written the examples from real (but old) support threads.
The agent replied, confidently, with the retired plan's price. The customer tried to buy it. I got an email asking why checkout didn't work.
That was the visible failure. The invisible one was worse: I checked the logs and found the agent had been pattern-matching my examples as facts for three days. Any question that was even loosely near an example got answered with details from the example — including numbers, timeframes, and policy specifics that were frozen in whatever moment I'd written them.
I had accidentally created a tiny, authoritative-looking knowledge base of stale facts and told the model "responses look like this." The model heard "these are true things."
The root cause, once I saw it: examples carry two kinds of information — style and content — and the model can't tell which parts you mean as demonstration and which as ground truth.
The fix: separate style from facts
I restructured the prompt into three explicit layers, and this is the part I'd hand anyone starting out:
LAYER 1 — FACTS (the only source of truth)
Current plans, prices, shipping windows, refund policy.
"Only use information from this section for any specific
number, date, price, or policy. If it's not here, say you'll
check and escalate."
LAYER 2 — STYLE EXAMPLES (demonstrations, not facts)
3–5 example exchanges. Prefaced with:
"These examples demonstrate TONE, LENGTH, and STRUCTURE only.
The prices, plans, and details in them may be fictional.
Never copy specific facts from these examples into replies."
LAYER 3 — ESCALATION RULES (short)
5–10 bullet rules for the genuinely hard cases: angry
customers, refund requests over $X, anything legal.
Two details that mattered more than I expected:
I deliberately made the style examples slightly fictional. Changed names, rounded numbers, invented order IDs. If the agent ever leaked example content into a real reply, it would be obviously wrong ("Order #12345") instead of subtly wrong (a real-looking but outdated price). Subtle wrongness is what gets you a refund dispute. Obvious wrongness gets you a caught bug.
I added one anti-example. A single "BAD response" with one line explaining why it's bad — in my case, an over-apologetic three-paragraph reply. Negative examples are underrated; they draw the boundary of the style region from the other side, and they cost almost nothing.
After the restructure, format compliance stayed at ~99%, tone stayed anchored, and fact hallucinations from examples went to zero in the following eight weeks. Not because the model got smarter — because the prompt finally told it which parts were a demonstration.
Rules I now follow when writing example-driven prompts
- Examples for style, explicit data for facts, never mix them. If an example contains a real number, that number will eventually be repeated to someone who shouldn't hear it.
- Audit examples like you'd audit dependencies. Every example is a frozen snapshot of your business. When prices change, examples go stale silently. I re-read mine on the first of every month — calendar invite, ten minutes, non-negotiable.
- Three to six examples is the sweet spot. Below three, the style isn't pinned. Above six, they start contradicting each other in subtle ways and you're paying tokens for noise.
- Include at least one hard case. Don't make all your examples happy-path. One example where the correct answer is "let me check and get back to you" teaches the agent that not-knowing is an acceptable output — that single example probably prevented more damage than everything else combined.
- Test with adversarial inputs before shipping. I threw 20 weird real customer emails at the new prompt, including ones near the edges of my examples. That's how I'd have caught the Starter Tier problem on day one instead of day three.
What I'd tell myself two months ago
Instructions aren't dead — my Layer 3 rules are instructions, and they're load-bearing. The lesson isn't "examples > rules." It's that the two do different jobs, and most broken agent prompts I've seen (including mine) are broken because one is doing the other's job. Rules trying to describe tone produce lawyer-speak. Examples trying to carry facts produce confident lies.
Give each layer its job, label the layers explicitly, and the model does what you meant instead of what you wrote.
All 100 prompts are in The Agent Prompt Vault — $3, lifetime updates. Steal the ones that fit your workflow.
Top comments (1)
Making example details obviously synthetic is the most reliable defense against this. If an example uses dummy identifiers like SKU-0000 or a fictional domain, any accidental leakage in production trips automated schema checks immediately instead of masquerading as valid business logic.