I started asking vendors for detail because the backlog was winning and every webinar promised "AI-powered QMS" with a sparkle emoji. To be fair, some of the marketing teams are enthusiastic — granted — but enthusiasm doesn't survive an audit or a notified-body question. In practice this means a vendor claim is worth exactly what they can document: which model, what data it was trained on, how decisions are surfaced, and where a named human reviewed and signed off.
Why the phrase is mostly marketing
"AI-powered" is a feature label, not a safety case. In five years of CE-marking under MDR 2017/745 I've learned notified bodies care about traceability and reviewability, not buzzwords. MDR makes manufacturers responsible for the QMS (see Article 10(9)(h) for the manufacturer's obligations to have and maintain a QMS). An eQMS that uses automated suggestions still needs to support:
- traceable authorisation and signature of decisions,
- versioned audit trails that show what changed and why,
- risk assessment of automated outputs (who accepts the residual risk),
- validation evidence that the automation behaves as claimed.
If the vendor cannot point to the model, the provenance of the data, and the human-in-the-loop workflow, you're buying a dashboard with a sparkle emoji. It looks pretty in a demo — it does not satisfy an auditor.
What I ask vendors to show me (concrete)
When evaluating an "AI" QMS, these are the concrete artefacts I request before anyone signs a contract:
- Model identification: name, version, and whether it’s an open-weight model, a hosted third-party API, or a vendor-trained proprietary model. If they use a third-party hosted API, which SLA and data retention policy applies?
- Training and data provenance: high-level description of the datasets used (not raw data), any medical device data included, and whether data was synthetic or real. This matters to bias and hallucination risk.
- Validation evidence: acceptance criteria, validation protocol, and test log showing the model's performance against those criteria in realistic QMS tasks (document classification, suggested CAPA root causes, change impact hints).
- Human sign-off workflow: where in the UI a human reviewer accepts/edits/rejects automated suggestions; time-stamped signatures tied to user accounts; and how that sign-off is recorded for audit.
- Explainability output: how the system explains a recommendation (e.g., which documents influenced the suggestion), not just a probability number.
- Change control and model versioning: how model updates are controlled, communicated, and validated within your QMS's change-impact process.
- Data handling and confidentiality: data flows, retention, and whether your input documents will be retained or used to re-train the model.
If a vendor dodges any of these or promises "we'll handle it later", they get a polite no from me.
Red flags vs acceptable answers
Red flags
- "Proprietary model" with zero technical detail and no model version.
- No documentation of human-in-the-loop; auto-applied changes without time-stamped review.
- Vague claims about "training on millions of documents" without provenance.
- No validation protocol or "we'll validate after you buy".
Acceptable (not perfect) answers
- The vendor names the model family and provides a validation protocol and sample results.
- The UI shows a clear "suggestion" state and requires a named user's digital sign-off before any document change is final.
- Model updates are treated as a change control item with documented impact analysis and re-validation.
How this actually plays out in audits
Notified bodies ask three things, consistently:
- Can you show me the decision trail? (timestamps, user IDs, what changed)
- Who validated the automation and when? (protocol, test evidence, sign-off)
- How do you manage model updates as a change to your software/service?
If you cannot produce those during an audit, expect a nonconformity on document control or software lifecycle. To be fair, some vendors already ship features that make this easy: one-search across the entire QMS, integrated traceability, and explicit "automated suggestion" flags in the record view. Those are the items that turn vendor blur into something auditable.
Practical checklist for procurement (copy-paste into your RFP)
- Require model identification and versioning in contract.
- Demand documented validation protocol and a sample validation report.
- Insist on a human-in-the-loop sign-off workflow with audit trail export.
- Specify change-control obligations for model updates (notification windows, staged rollout, rollback).
- Ask for data-use and retention statements — no training on your documents without explicit consent.
This is not legal advice; it's what I ask for when my notified body is two months away and I need the audit trail to be surgery-ready.
Where QMS software can help (and where it can't)
Helpful:
- Native workflow integration that ties automated suggestions into change control and CAPA workflows.
- Automated CAPA suggestion draft that the owner edits and signs — useful if it's traceable and reviewable.
- AI-powered search that surfaces evidence during an audit, provided the results are reproducible and explainable.
Not helpful:
- Fancy dashboards showing "AI impact" metrics with no exportable audit trail.
- Claims that "AI reduces your workload" without showing how residual risk is accepted and recorded.
Connected workflow is the only way AI assistance becomes compliance-assistive: the suggestion must live in the same traceable record as the sign-off.
Final thought
We are not banning automation; we are asking for accountability. Name the model. Show where the human signs off. Otherwise it's marketing copy dressed up as innovation.
What have you asked for — or been refused — when buying an "AI-powered" QMS?
Top comments (0)