Plain chain-of-thought hands the model a few correct worked examples and hopes it generalizes the right steps. It's genuinely strong — but I kept watching it walk into the same trap on multi-step arithmetic: adding when it should subtract, jumbling the step order, grabbing the wrong number. The problem is that a correct-only prompt draws just one side of the line. The model sees what good reasoning looks like and never what bad reasoning looks like. Contrastive CoT (Chia et al., 2023) closes the boundary, and I built a live prompt builder to prove it. Here's how it works.
The slip a correct-only prompt leaves on the table
Few-shot CoT is a handful of questions, each answered with a step-by-step rationale, then the target. On a 3-step problem — "23 croissants, sold 15, baked 30 more" — the model that only saw clean chains still fumbles: it adds every number (68) instead of subtracting the outflow first (38). The examples never demonstrated that the "add everything" path is wrong, so nothing in the prompt rules it out.
// STEP 1 — ordinary few-shot CoT: correct rationales, then target.
const shots = examples.map(e =>
`Q: ${e.q}\nA: ${e.rationale} So the answer is ${e.answer}.`
).join("\n\n");
const prompt = `${shots}\n\nQ: ${target}\nA:`;
// works... until a multi-step target triggers a predictable slip.
Generate a negative by corrupting a correct chain
You don't hand-write wrong answers — you break a right one in a controlled way. The paper corrupts the bridging objects and reasoning; in my demo that's three cheap, automatic transforms: flip an operator, shuffle the step order, or swap two numbers. Each yields a plausible-looking but invalid chain.
function corrupt(rationale, kind) {
if (kind === "operation") return flipAnOperator(rationale); // − becomes +
if (kind === "order") return lumpStepsOutOfOrder(rationale);
if (kind === "entity") return swapTwoNumbers(rationale);
}
// operation: "40 − 12 = 28 ; 28 + 25 = 53" -> "40 + 12 = 52 ; 52 + 25 = 77"
The label is load-bearing
This is the single most important detail. A corrupted chain shown without a "this is wrong" marker just teaches the mistake. Each example becomes a pair — the correct explanation, then the wrong explanation explicitly flagged as the path to avoid — and that pairing is what draws the boundary.
const pair = e => ({
q: e.q,
correct: { rationale: e.rationale, answer: e.answer },
wrong: { rationale: corrupt(e.rationale, kind),
label: "Wrong explanation (do NOT reason like this)" }
});
// unlabeled wrong == teaching the error. the label flips it into a lesson.
Assemble the prompt — one call, no fine-tune
Interleave per example: question → correct explanation → labelled wrong explanation. Then the target with a short guard line. That's the whole technique — a richer prompt, still one inference call, no extra model. This is exactly what the live builder assembles as you toggle it.
const block = p =>
`Q: ${p.q}\n` +
`Correct explanation: ${p.correct.rationale} So the answer is ${p.correct.answer}.\n` +
`Wrong explanation (do NOT reason like this): ${p.wrong.rationale}`;
const prompt = examples.map(pair).map(block).join("\n\n") +
`\n\nQ: ${target}\nA: (reason step by step; avoid the mistakes shown above)`;
Where it earns its place
Contrastive CoT helps most on multi-step arithmetic and symbolic reasoning, where errors are systematic and easy to construct as counter-examples. It's a strict upgrade of plain CoT — same worked examples plus their labelled negatives — and it stacks with self-consistency: contrast lowers each sample's error rate, then you vote across samples. Keep the negatives clearly labelled and use one or two per example; too many, or unlabeled, and you risk teaching the very mistake you meant to rule out. Showing the error, as an error, beats hoping the model never makes it.
Flip Contrastive on and watch the assembled prompt rebuild — and the answer flip from 68 to 38:
https://dev48v.infy.uk/prompt/day47-contrastive-cot.html
Top comments (1)
I found the concept of Contrastive Chain-of-Thought fascinating, particularly how it addresses the limitation of plain chain-of-thought by providing a labelled wrong explanation to draw the boundary. The example of corrupting a correct chain by flipping an operator, shuffling the step order, or swapping two numbers is a great way to generate plausible-looking but invalid chains. I'm curious to know more about how the choice of corruption method affects the model's performance - for instance, do certain types of corruption (e.g. operation vs order) have a greater impact on the model's ability to learn the reasoning boundary?