Ask a model a tricky problem cold and it does what it always does — grabs the nearest familiar pattern and runs with it. On problems that have a well-known trap, the nearest pattern is exactly the wrong one.
Try this one: how many 3-digit numbers have all distinct digits? The clean, tempting answer is 10 × 9 × 8 = 720. It's wrong. The first digit can't be 0 (or you're counting things like 072, which isn't a 3-digit number), so it's 9 × 9 × 8 = 648. A model answering from a cold start reaches for the clean product and walks straight into the trap.
Few-shot prompting is the classic fix: paste in a couple of solved examples so the model sees the right method before it hits the target. It works. But the examples have to come from somewhere — you need relevant, correct, well-chosen exemplars on hand for every new problem, and you have to keep an example bank around. On a brand-new or highly varied problem, you often don't have good ones. And an irrelevant or wrong example actively hurts.
Analogical Prompting (Yasunaga et al., 2023) flips who writes the examples. Instead of you supplying exemplars, you tell the model to recall and write out a few relevant solved problems itself, then solve the target by analogy to them. It's few-shot where the model curates its own shots, pulling from the millions of solved problems it saw during pretraining.
Here's the whole technique as one prompt:
Problem: {your problem}
# Relevant problems:
Recall THREE distinct problems you know how to solve that are
similar to this one. State each problem and its full step-by-step
solution.
# Solution to the original problem:
Using those as analogies, solve the original problem.
Run our distinct-digits problem through that and the model warms itself up first. It recalls: "how many 2-digit numbers have distinct digits? First digit 1–9 (can't be 0) = 9 ways; second is 0–9 except the first = 9 ways, so 81." And: "3-letter codes with no repeat = 26 × 25 × 24, each pick drops one option." Now the leading-zero rule and the drop-one pattern are both sitting in context. Solving the target by analogy is easy: leading digit 9 ways, then 9, then 8 → 648. Correct.
Why does self-generating examples help when the knowledge was already in the model? Because a cold prompt never activates the right part of it. The recall step pulls the relevant method into the context window, and since generation is autoregressive, every token of the target solution is then conditioned on those freshly-written examples. The exemplars act like a self-built few-shot prefix that lays down the correct reasoning pattern right before it's needed.
A few things that make it work in practice:
Give each example its own reasoning. The exemplars are useful because they carry a worked method, not just a final answer. Ask the model to solve each recalled problem step by step — the method is what transfers to the target. This is analogical prompting done as intended: self-generated examples, each with a chain of thought.
Three to five diverse examples. One is a weak signal and often just restates the trap. A dozen invites drift and near-duplicates. Ask explicitly for distinct problems — different numbers, framings, sub-methods — so you cover more of the method space instead of the same trick three ways.
Watch out for bad self-examples. The exemplars are only as good as the model's own generation. A self-generated example can be wrong, irrelevant, or hallucinated — and because the solution reasons by analogy to it, a bad exemplar drags the answer down with it. Have the model verify each example's answer and relevance, and keep a plain direct-solve fallback for when the recalled examples don't actually match.
It has a close cousin worth knowing: Generated Knowledge Prompting (Liu et al., 2022) does the same move but recalls facts and principles instead of solved problems, then answers using them. Use analogical when the gap is a missing method; use generated-knowledge when the gap is missing background knowledge. Same engine, different fuel.
So when do you pick this over hand-written few-shot? Reach for analogical when you have no good examples on hand, when the problem space is wide enough that no single fixed example set fits every input, or when you just don't want to maintain an example bank — the model tailors fresh exemplars to each call. Stick with hand-written few-shot when you need a specific output format or have vetted gold examples whose correctness you can't leave to the model.
I built an interactive walkthrough — pick a target problem and watch the cold solve fall into the trap while the analogical track self-generates examples and gets it right: https://dev48v.infy.uk/prompt/day29-analogical.html
Top comments (0)