The Test Passed All Three Bugs. That's the Problem.

Three deliberately broken shipping functions. The same test passes every one.

That result should make a green test badge feel less reassuring.

Here is a small experiment you can run in Node.js. The business rule is explicit: shipping costs 500 cents below a subtotal of 5,000 cents, and is free at or above that threshold. All inputs in the experiment are valid nonnegative integer subtotals.

A plausible test with a large blind spot

import assert from "node:assert/strict";

// Amounts are integer cents. This example uses valid subtotals only.
const variants = {
  reference: s => s >= 5000 ? 0 : 500,
  wrongBoundary: s => s > 5000 ? 0 : 500,
  alwaysFree: s => 0,
  wrongFee: s => s >= 5000 ? 0 : 5000
};

for (const [name, shipping] of Object.entries(variants)) {
  assert.equal(shipping(6000), 0);
  console.log(name, "PASS");
}

The output is four passes. The correct implementation passes. So do three incorrect ones: a threshold off by one, free shipping for every order, and a fee ten times too large below the threshold.

Nothing is wrong with assert.equal. The test asks only about an order that qualifies for free shipping. All four implementations answer that particular question correctly.

Ask the test to distinguish the alternatives

Keep the 6,000-cent case, then add observations at the boundary and immediately below it:

assert.equal(shipping(4999), 500);
assert.equal(shipping(5000), 0);
assert.equal(shipping(6000), 0);

Running those assertions against the four implementations produces this matrix:

Implementation	Original check	Three-case check
Correct rule	Pass	Pass
> instead of >=	Pass	Fail
Always free	Pass	Fail
Wrong fee	Pass	Fail

These are results from the local example, not measurements of a model or a production checkout. The three-case check detects these three changes. It does not establish complete input validation, tax correctness, currency handling or checkout reliability.

The same assistant can miss the rule twice

When a coding assistant proposes both an implementation and its tests, inspect which examples it chose. If both artifacts share an omitted boundary, agreement between them tells you little about that boundary.

There is no model comparison hidden in this experiment. A person can write the same weak test. The useful question is whether the test can reject a believable wrong implementation.

A more useful review request is:

Name three plausible mistakes in this rule. Change the implementation to each one separately, then show which assertion fails.

That asks for evidence of discrimination. “Add more tests” can produce a longer suite of the same happy path.

You can start without another test framework

The deliberate changes here are a small manual exercise. Automated mutation testing applies this idea more broadly. Stryker's documentation explains the difference between changes that tests detect and changes that survive.

Start with a consequential rule whose expected outcomes you can explain independently: a shipping threshold, a permission decision or an import rejection. Keep the expected behavior separate from the implementation under review.

If no one can explain why an expected value is correct, asking another model to repeat it does not settle the requirement.

Which believable bug would your current test suite accept without turning red?