Few-shot chain-of-thought is the reliable way to get a model to reason: show it a handful of worked examples and it copies the habit. The catch is where those demonstrations come from. Manual few-shot CoT (the hand-crafted kind) makes you author each step-by-step example — slow to write, doesn't scale to new task families, and it quietly bakes in one person's phrasing and blind spots. Auto-CoT (Zhang et al., 2022) keeps the accuracy and throws out the hand-labour, by having the model build its own demos. I built the whole pipeline in-browser to watch it work; here's the shape of it.
The obvious automation is the wrong one
If you want demos automatically, the tempting move is retrieval: pick the pool questions most similar to your target and let plain zero-shot CoT ("Let's think step by step.") write a rationale for each. It's automatic — but zero-shot CoT makes correlated mistakes on one hard question-type, so a similarity-picked set can be several demos of the same misleading kind, amplifying one error instead of diluting it. Similarity is exactly the trap.
Cluster first, so the demos span different shapes
Auto-CoT's fix is diversity by construction. Embed every question in the pool, then cluster them — k clusters means k different kinds of problem. The variety is the whole point: no single hard type can dominate the demo set, and a stray wrong rationale is out-voted by the diverse rest. In the demo this is real Lloyd's-algorithm k-means over a toy concept-axis embedding (production Auto-CoT calls Sentence-BERT), nothing hard-coded.
function kmeans(vecs, k){
let cent = farthestFirstInit(vecs, k); // deterministic spread
let assign = vecs.map(() => 0);
for (let iter = 0; iter < 20; iter++){
let moved = false;
vecs.forEach((v,i) => { // assign to nearest centroid
const c = argmin(cent.map(m => dist2(v,m)));
if (c !== assign[i]){ assign[i] = c; moved = true; }
});
cent = recomputeCentroids(vecs, assign, k);
if (!moved) break; // converged
}
return { assign, cent };
}
One representative per cluster
From each cluster take the question closest to its centroid — the most typical member — with a light heuristic preferring concise ones (the paper caps question length and rationale steps). One representative per cluster is what gives you exactly k diverse demonstrations, one per problem shape.
Let the model write each rationale
For every representative, ask the model itself to produce the reasoning with the magic zero-shot line, and read off its answer. No human writes these — the model authors its own demos.
async function autoDemo(question){
const rationale = await llm(`Q: ${question}\nA: Let's think step by step.`);
return `Q: ${question}\nA: Let's think step by step. ${rationale}`;
}
const demos = await Promise.all(reps.map(r => autoDemo(r.q)));
Assemble once, reuse for every query
Concatenate the auto-built demos, append the new question with a bare "Let's think step by step.", and let the model complete it. The diverse demonstrations condition it to reason the same way on your target. And the demos are built once from the pool, then reused for every new question.
function autoCoTPrompt(demos, target){
return demos.join("\n\n") + `\n\nQ: ${target}\nA: Let's think step by step.`;
}
The lesson the paper drives home is counterintuitive: diversity of demonstrations matters more than similarity. Misleading demos hurt, and variety immunises against them — which is why clustering the pool beats retrieving lookalikes. It's prompt-construction only, no fine-tune, and it sits distinct from contrastive-CoT (which adds labelled wrong chains) and buffer-of-thoughts (which reuses distilled reasoning templates, not clustered example questions). It matches manual few-shot CoT on reasoning benchmarks while eliminating the labelling cost, and beats naive similarity-sampling by immunising against clustered zero-shot errors. Reach for it when you have a recurring task family and a pool of example questions but no appetite for hand-writing demos; skip it for a genuine one-off with no pool. Change k in the live demo and watch the demo set re-diversify — try k=1 to see diversity collapse:
Top comments (0)