If you're not a lawyer, here's a practical gut-check for any AI you put into a QMS or quality workflow. Ask these two plain questions. If either answer is yes, stop what you're doing and fix it.
- Does any AI in your stack talk directly to a human without clearly saying "I am AI"?
- Does any AI in your stack create content that gets published externally (customer emails, supplier letters, website copy, regulatory submissions) without a named person reviewing and sign-off?
If the answer to either is yes, that is the single, highest-priority thing to fix. Everything else — model choice, governance committee composition, fancy mitigation matrices — is secondary until you close these two holes.
Why these two questions matter
I run supplier quality at a contract manufacturer. We automate incoming inspection, COA verification and CAPA flags with a Python/Postgres/Grafana stack. I’ve seen many well-meaning automation projects where an AI suddenly became the frontline communicator: it triaged supplier evidence requests, drafted CAPA descriptions, or suggested wording for change notifications. Two failure modes appear repeatedly:
- People assume "smart-sounding" equals "safe." An AI can produce plausible text that sounds authoritative but lacks traceability or judgement. That's dangerous when it feeds a regulator or a customer.
- Automated outbound content without human review creates legal and compliance exposure. If an AI writes a corrective action plan or a supplier-facing containment instruction and no named person reviewed it, who signs the record? Regulators and notified bodies will expect a clear accountable person.
Standards like ISO 13485 and 21 CFR Part 820 require traceable, reviewable records and defined responsibilities. If your machine produces "records" without a human reviewer on the record, you're not meeting the spirit of those requirements even if your vendor marketing says otherwise.
Simple, non-lawyer fixes you can implement today
I like fixes that are small, visible, and auditable. Here’s a checklist you can run in hours or days, not months:
- Label outward-facing messages: prepend or append "Draft generated by AI" or "AI-assisted" to any supplier/customer-facing text.
- Block autonomous external publishing: configure systems so AI output is saved as a draft that cannot be sent externally until a named user approves.
- Require named human approval: add a mandatory sign-off field (full name, role, timestamp) before any AI-generated content becomes a QMS record or external communication.
- Log provenance: store the AI prompt, model version, response, and the human reviewer’s comment in the record history.
- Add a short risk assessment to your change control: low-effort but explicit — what can go wrong if the AI is wrong? Who is liable?
- Update SOPs and training: make "AI-assisted content" a controlled document type. Train the people who will be reviewing.
- Monitor and audit: run weekly reports for the first quarter showing how many AI drafts were approved, by whom, and how many were edited.
These are low-friction controls that create traceability and enforce human responsibility. They map cleanly into existing QMS processes: document control, change control, CAPA, and management review.
Why "human in the loop" is not a rubber stamp
There’s an attractive slogan you may have seen: "AI proposes, human approves and signs." That is the right principle, but it only works if the approval is meaningful.
A checkbox that says "I accept" with no evidence of actual review is not sufficient. The human reviewer must have:
- Access to the AI prompt and full response
- The ability to edit the draft before signing
- A recorded justification if they approve without edits (why does the reviewer believe the AI output is correct?)
Treat human approval as a record. Make it easy to show this to an auditor or notified body: they will expect reviewability and traceability, not a faith-based claim.
A short technical pattern that works
From my automation work, this pattern is practical and keeps workflows connected:
- AI engine produces draft -> stored in Drafts table with metadata (prompt, model, timestamp)
- Reviewer UI shows diff: AI draft vs. edited final
- Reviewer must add a comment and sign (electronic signature or tracked approval)
- On approval, final is promoted to Record/External message, with provenance attached
- Any downstream actions (CAPA creation, supplier notification) link back to the signed record
This keeps the conversation and decision-making inside your connected workflow so you don't create siloed, unverifiable outcomes.
A word on regulation and marketing hype
Vendors love to brand their features as "AI-driven CAPA assistance" or "auto-closure" — buyer beware. Beta features are beta; small vendors present a legitimate supply risk. The relevant regulatory bodies care about traceability and accountability, not marketing tags. If a feature bypasses named-person review, that’s a compliance red flag, irrespective of the model or API your vendor uses.
If you use tools that connect conversation and decision records (that’s why I often talk about connected workflow tools), check how they capture reviewer identity and provenance before you enable any outbound automation.
I work on qmsWrapper and I’m sharing an honest read of where such tools may not fit every CMO or supplier-side use case.
What’s the one simple control you added last year that actually saved you time or risk when AI produced a bad draft?
Top comments (0)