Five Users Is a Stop Rule, Not a Sample Size

Every front-end team has inherited some version of this: "we don't need real usability data, Nielsen said five users is enough." The recommendation is real. The way it gets quoted into standups is not. Five users is a stopping rule for problem discovery. It is not a sample size for anything you want to measure, and the difference is arithmetic you can implement in about twenty lines.

Conceptual AI-generated cover for this article, composited with the unchanged official Design Den Solutions logo. The logo artwork itself is not AI-generated.

The model behind the number

Jakob Nielsen's original claim (Why You Only Need to Test with 5 Users, NN/g, 18 March 2000) is a coverage model, not a headcount heuristic:

P(n) = 1 - (1 - L)^n

• n = testers in the round
• L = the proportion of the total problem set that one tester finds
• P = cumulative proportion of the problem set found after n testers

Nielsen used L = 0.31, which lands at ~85% at n = 5. He was explicit that this is a cost/benefit stopping point, not a completeness claim: "The best results come from testing no more than 5 users," and in the same article, "You need to test with at least 15 users to discover all the usability problems." His 2012 restatement (How Many Test Users in a Usability Study?) keeps "test 5 users in a qualitative usability study" and gives the reason not to inflate n: "ROI drops like a stone with a bigger N."

const coverage = (L, n) => 1 - (1 - L) ** n;

for (let n = 1; n <= 8; n++) {
  const prev = n === 1 ? 0 : coverage(0.31, n - 1);
  console.log(
    `n=${n}  total=${(coverage(0.31, n) * 100).toFixed(1)}%  ` +
    `this tester adds ${((coverage(0.31, n) - prev) * 100).toFixed(1)} pts`
  );
}
// n=1 total=31.0% this tester adds 31.0 pts
// n=3 total=67.1% this tester adds 14.8 pts
// n=5 total=84.4% this tester adds  7.0 pts
// n=8 total=94.9% this tester adds  2.3 pts

That marginal column is the entire argument. Tester five returns roughly a fifth of tester one's yield. The model assumes equal findability across problems, which is false in practice: a label nobody clicks is nearly invisible to every tester, while a broken button is found by the first person who touches it. So L is not measurable before the round, and 85% is a planning figure, not a guarantee. At L = 0.20, five testers cover 67.2%; at L = 0.15, 55.6%.

Implement the stop rule, not the magic number

The useful engineering artefact is a uniqueness curve over the problems you have actually logged:

function shouldStop(problemLog, testersNeeded = 2) {
  const firstSeen = new Map();
  problemLog.forEach((p, i) => { if (!firstSeen.has(p.id)) firstSeen.set(p.id, i); });

  const newPerTester = Array.from({ length: problemLog.length }, (_, t) =>
    [...firstSeen.values()].filter((i) => i === t).length
  );

  const tail = newPerTester.slice(-testersNeeded);
  return {
    newPerTester,
    stop: tail.every((x) => x === 0),
    reason: `last ${testersNeeded} testers surfaced ${tail.reduce((a, b) => a + b, 0)} new problem classes`
  };
}

"Stop" is not "five reached". It is "the last two testers produced no problem class you had not already recorded". On a checkout flow with one obvious blocker, that can happen at three. On a genuinely untested dashboard, it may not happen until seven. Five is the expected cost of a round, not the exit condition.

What five people cannot tell you

This is where teams write a number into a deck that the data cannot carry.

Three of five testers abandon the enquiry form. Someone reports "60% of users fail the form". Five is not a survey; the correct statement is an interval.

function wilson(k, n, z = 1.96) {
  const p = k / n, d = 1 + (z * z) / n;
  const c = (p + (z * z) / (2 * n)) / d;
  const h = (z * Math.sqrt((p * (1 - p)) / n + (z * z) / (4 * n * n))) / d;
  return [Math.max(0, c - h), Math.min(1, c + h)];
}

wilson(3, 5);   // [0.23, 0.88]  -> "23% to 88%"
wilson(12, 20); // [0.39, 0.78]

A 65-point-wide interval is not a conversion finding. To hold ±10 points around a 60% success rate you need n = z²p(1−p)/e² ≈ 93. If the goal is a rate — task success, completion time, benchmark against a competitor — that is a quantitative study with a different budget, and no amount of qualitative rounds substitutes for it.

Two more things five users will not do:

• Prove the redesign worked. Before/after on five people each is anecdote with a delta.
• Establish accessibility conformance. WCAG is a standard you audit against, not a headcount. Contrast under SC 1.4.3 is 4.5:1 for normal text and 3:1 for large text, computed from relative luminance. Five sighted testers will not tell you whether your orange-on-white passes, and neither will fifty.

And one that is easy to get wrong: five per study is not five per audience. The NN/g guidance is "you'll need close to 5 users per group". Split across two personas, each group gets 2.5 people and the model's single-population assumption no longer applies.

Rounds beat bigger N

A second round of five after the fixes is not a repeat of the first. It runs against a changed interface, so it can surface problems the first round structurally could not reach — the flow you just unblocked, the copy you just rewrote, the regression you just shipped. One round of ten would find 97.6% of the problems that existed at the start, and 0% of the ones created by your fixes. Iteration is the point; Nielsen's own phrasing is to "plan for a lot of iterations".

Make each session produce a bug, not a vibe

Instrument the same fields every time so the round aggregates:

{
  "task": "find out whether installation is included",
  "tester": "T3",
  "completed": false,
  "helpRequested": true,
  "timeOnTaskSec": 143,
  "firstClick": ".pricing-card",
  "quote": "I can't tell if they fit it or just sell it",
  "problemIds": ["pricing-scope-ambiguous"],
  "severity": "serious"
}

Severity rubric that survives review: blocker = cannot complete without help; serious = completes with visible struggle or a wrong assumption; cosmetic = preference, no task impact. Then write the finding the way you would write a defect: task, page or selector, observed action, user's stated goal, consequence. pricing-scope-ambiguous with two testers and a quote is actionable. "Users find pricing confusing" is not.

Two operational constraints worth putting in the test script before you invite anyone:

1. Consent and data. Record only with explicit consent, store the minimum, and never let a tester enter real personal data, make a payment, or fire a live enquiry into your own production system. Use a staging environment or stop at the boundary.
2. Recruit for the actual population. Colleagues share your mental model and will silently repair your copy for you. Five strangers beat five teammates every time.

The honest summary

Five users is the cheapest round that reliably produces a ranked fix list, because the marginal yield collapses right after it. Use it to decide what to build next. Do not use it to report a percentage, to certify accessibility, or to claim a lift. Those are different studies, and the arithmetic above will tell you the sample size you actually need before you promise anyone a number.

---

References

• Nielsen, J. Why You Only Need to Test with 5 Users. Nielsen Norman Group, 18 Mar 2000.
• Nielsen, J. How Many Test Users in a Usability Study? Nielsen Norman Group, 3 Jun 2012.
• W3C, Understanding SC 1.4.3 Contrast (Minimum), WCAG 2.2.

Disclosure: this article was prepared with AI assistance and reviewed for accuracy by Design Den Solutions. The cover image is a conceptual AI-generated illustration with the official company logo composited unchanged. All percentages come from the discovery model and the code shown above, which was executed before publishing; no client data or user metrics are represented.

Design Den Solutions builds and reviews websites and apps for businesses in Mohali and the Tricity — web development and UI/UX services.