Where usability files actually break — and what I do about it

I stopped treating the usability file as a compliance artifact years ago — it’s where product design and real people collide. When it’s done badly, the consequences are more than an audit finding: increased use errors, higher CAPA rates, and slower time‑to‑market. Below are the concrete places I see Human Factors (HF) work fail in Class II products and pragmatic steps I've used to keep the usability file useful, auditable, and tied into risk control.

Why the usability file matters (beyond the checkbox)

Standards and guidance (IEC 62366‑1, FDA Human Factors Guidance) frame the work, but the goal is safer use. The usability file should:

• Show you understood who will use the device and how (context of use).
• Demonstrate that you analysed use‑related hazards and applied risk controls (link to ISO 14971).
• Provide evidence that controls work (formative + summative testing). If those links are missing, you may have a 40‑page file that still doesn’t prove safety in the real world.

Where designs and files tend to break (concrete failure modes)

These are patterns I repeatedly encounter in reviews and audits:

• Scoping too late
  • HF starts after detailed hardware/GUI decisions are locked. Result: recommendations that are impractical to implement.
• Shallow context-of-use
  • “Nurses” or “patients” listed as users without role breakdown, environment, training level, or typical workload.
• Task analysis that’s too abstract
  • High‑level steps (“prepare device”, “administer dose”) without identifying critical subtasks and error‑prone interactions.
• Formative testing that isn’t formative
  • Small, unstructured sessions that produce observations but no iterative design response.
• Summative testing that lacks realism
  • Lab conditions, scripted tasks, or non‑representative users — testers who aren’t the real world’s problem.
• Poor linkage to risk management
  • HF findings aren’t traced to hazards, risk controls, or residual risk acceptability criteria.
• Documentation gaps
  • Missing test scripts, consent forms, moderator notes; unclear pass/fail criteria leading to auditor questions.
• No feedback loop
  • Post‑market complaints and CAPAs don’t feed back into the HF process or change the usability file.

Practical fixes that stuck for me

I work in a 200‑person company with a small QA/RA team, so these are pragmatic, low‑overhead practices I’ve adopted:

• Start HF at requirements
  • Make HF inputs part of the initial requirements spec: intended users, environment, frequency, and critical tasks. This prevents late scope creep.
• Define critical tasks explicitly
  • For each task, document success criteria and what constitutes a use error. Use a one‑line “criticality” tag so engineers can see risk immediately.
• Recruit representative users early
  • Don’t rely on convenience samples. If you can’t access clinicians, simulate crowdwork with validated scripts only as a last resort.
• Use realistic prototypes in formative testing
  • Even cardboard interfaces can expose major interaction failures. Test under realistic time pressure, ambient noise, and multitasking scenarios.
• Make summative tests realistic and focused
  • Test only critical tasks needed for validation; scripts should reflect typical and foreseeable misuse scenarios. Predefine pass/fail.
• Treat findings as inputs to RCA
  • Use the same RCA flow we use for CAPA: define problem, collect data (HF observations), identify causal factors, identify root causes, implement solutions. This keeps HF findings actionable and traceable.
• Link HF evidence to risk controls in the QMS
  • Map each HF mitigation to the associated ISO 14971 risk control record. That speeds audits and prevents “orphan” usability recommendations.
• Keep the usability file lean and evidence‑forward
  • Executive summary, context‑of‑use matrix, task‑hazard map, tests and results, and traceability matrices. Store raw data (videos, transcripts) separately but reference them.

Documentation and audit hygiene

Auditors want to see that you performed the work and that conclusions are defensible:

• Include test scripts, recruitment criteria, moderator instructions, and raw outcomes.
• Show how each usability issue was triaged: accepted change, mitigated by labeling, or accepted residual risk — and who approved it.
• If you made late UI changes, include a re‑test plan and evidence.
• Version the usability file and link it to design change records so change control is clean.

Connecting HF to CAPA and PMS

Human factors problems will show up in CAPA queues and vigilance data. Don’t silo them:

• Feed use‑related complaints into HF scoping decisions.
• If recurrent issues appear, trigger a formal HF re‑evaluation rather than ad‑hoc fixes.
• Where possible, automate traceability so HF findings create linked records in your QMS and, if needed, into CAPA or change control workflows.

Final note

Good HF work is iterative, integrated, and visible. The usability file shouldn’t be a post‑hoc justification — it should be the narrative of how you discovered risk, fixed it, and verified the fix.

How do you balance the need for realistic summative testing with the budget and recruiting limits of a small medtech company? I'd love to hear practical approaches that worked for teams operating at our scale.