DEV Community

Cover image for Few-Shot Prompting: Show, Don't Describe
Jeff Reese
Jeff Reese

Posted on • Originally published at purecontext.dev

Few-Shot Prompting: Show, Don't Describe

I once spent longer than I care to admit trying to describe exactly how I wanted ChatGPT to format a list of meeting notes. I explained the sections I wanted, the bullet style, where quotes went, how to mark action items. The output kept almost getting it, then missing one detail and drifting somewhere else. I got frustrated, deleted my instructions, pasted two real examples of the format I wanted, and asked for the same treatment on the new input. It nailed it on the first try.

That was the moment I stopped describing and started showing.

In the first series we covered the three things that make a prompt work: context, task, format. Describing is how you get further with all three. Showing is the lever you reach for when describing stops working.

Why examples beat explanations

When you describe what you want, the AI has to translate your words into a pattern, then try to reproduce that pattern. Every step of that translation is a place the pattern can slip.

When you give it examples, the pattern is already there. The AI is extremely good at extending a pattern it can see. It does not have to guess what you meant by "formal tone with light personality." You showed it.

This is called few-shot prompting. One example is one-shot. Two or more is few-shot. No examples is zero-shot. The names aren't important; the shift in approach is.

Where showing helps the most

Anything where the shape of the output matters more than the content. Structured data extraction, format-sensitive tasks, stylistic consistency, edge cases you know about.

Say you are pulling company names and dollar amounts from a pile of unstructured notes. You could describe: "Extract the company name and the funding amount from each entry, return them as JSON." You might get something close, but edge cases will bite. What if the note mentions two companies? What if the amount is a range? What about entries with no amount?

A better approach:

Note: "TechCo raised $5M Series A in March"
 {"company": "TechCo", "amount": "$5M"}

Note: "Acme announced funding between $10M and $15M"
 {"company": "Acme", "amount": "$10M-$15M"}

Note: "Quantum Labs held a press event"
 {"company": "Quantum Labs", "amount": null}

Note: [your input]

Enter fullscreen mode Exit fullscreen mode

Three examples, three edge cases, and the AI now has a schema it can extend. You did not have to explain the rules. The examples carry them.

How many examples is typically enough

I have found that usually two to five is the sweet spot for most tasks. One example tells the AI the rough shape; the second confirms which parts of the first were pattern and which were incidental. By the third, the pattern is usually locked in. Past five, you are mostly adding noise and eating up your context window.

There are cases where one carefully chosen example works, and cases where you need more variety to cover the edges. The question to ask: do my examples cover the range of situations the AI will see? If yes, stop adding. If no, add one that fills the gap.

When showing backfires

The trap with few-shot is that the AI is so good at pattern-matching that it will over-match if you are not careful.

If all three of your examples happen to be from tech companies, the AI may start assuming the output should always be about tech companies, even when the input is about a restaurant. If all your examples have exactly two action items, the AI will start producing exactly two action items whether the input warrants it or not. That one bites.

The rule: your examples should vary on the dimensions that are not part of the pattern, and stay consistent on the dimensions that are. If format is the pattern, vary the content wildly. If tone is the pattern, vary the topic and length. The AI should have no way to mistake incidental variety for instruction.

The thing that is easy to miss

Few-shot prompting is not fine-tuning. The examples live in your prompt, not in the model. Every time you open a new conversation, you start from zero. The AI does not remember the examples you showed it yesterday. It does not "learn" your format over time from being shown it.

This means:

  • You need the examples every time, in every new conversation, until you paste them in
  • Examples count against your context window on every call
  • You're paying tokens to re-establish the pattern each session

For a one-off task, this is fine. For a recurring pattern, you are better off saving the example block as a snippet you can paste into new conversations, or eventually moving the work to an API call where the prompt template is persistent.

The reflex

Stop describing formats. Start showing them.

The next time you catch yourself writing a long paragraph explaining the exact structure you want the output to have, delete the paragraph. Paste two or three examples of the format instead. The output will be closer to what you wanted, you will get there faster, and the prompt will be shorter.

The move: if I can show it, I should show it.

Tomorrow: how to get an AI to reason through a problem instead of just answering it.

If there is anything I left out or could have explained better, tell me in the comments.

Top comments (0)