Right, so it's in force now. Last Friday I asked our product and legal leads whether the AI assistant embedded in our QMS "discloses itself properly" under the new EU AI Act transparency duties. The answer: "probably fine." That was not comforting. We're a 200-person company with Class II devices — "probably fine" is literally a compliance risk.
I write this from the practical side: we've got an AI-powered Smart Form Assistant that suggests text for CAPA descriptions and helps triage nonconformances. Engineers and QA staff rely on it during routine work. That makes the disclosure question operational, not theoretical.
What the transparency duties mean in practice (short)
The high-level requirement from the AI Act that matters to product teams is straightforward: users must be informed when they are interacting with an AI system, and they should get understandable, relevant information about the system’s capabilities and limitations. How that is implemented in your product/UI/SOPs is where the rubber meets the road.
That doesn't mean a buried paragraph in your terms-of-service — auditors will expect clear, discoverable disclosure and traceable evidence that users were informed.
Why "probably fine" isn't a defensible position
- It's not documented: "probably" leaves no audit trail, no procedure update, no evidence.
- It ignores context: different users (engineers, clinicians, suppliers) have different expectations and risk profiles.
- It ignores outputs: if the assistant drafts or materially influences decisions (e.g., CAPA root cause text, MDR reporting language), the disclosure and mitigation must be stronger.
If an inspector asks "how did you inform the user?" you need to show where, when, and what was communicated — and how users acknowledged or could access that info.
A pragmatic checklist I used to move from "probably" to "provable"
I drafted a working checklist we could actually implement in a week. Use it as a starting point — adapt to your product and risk profile.
- Map AI touchpoints
- Identify every UI element, API, and integration that uses the assistant (chat widgets, inline suggestions, email drafts, exported reports).
- Classify user groups and use-cases
- Who interacts with the assistant? Clinicians? Suppliers? Internal QA? Each group may need different wording or more explicit warnings.
- Add clear, visible disclosure
- Short label where interaction begins (e.g., “AI Assistant — suggestions generated by an AI”).
- Short banner or tooltip with the essentials: what it does, its limitations, and who to contact.
- Capture interaction evidence
- Log timestamps of interactions, suggestions presented, and whether the user accepted/edited/rejected them.
- Store a digest in your DMS with immutable metadata for audit.
- Update SOPs & training
- Document how staff should use AI outputs (review, validate, keep records).
- Include example wording for reviewers and a mandatory checkbox or acknowledgment where appropriate.
- Provide model info (at an appropriate level)
- Provider, model version/date, and a plain-language description of known weaknesses (hallucination risk, not a substitute for professional judgment).
- Update risk assessments
- Re-run risk and usability analyses where the assistant affects decision-making, and add mitigations (e.g., human review gates).
- Legal signoff & logging of policy changes
- Record the legal/regulatory review and keep a record of the disclosure wording you used over time.
Implementation patterns that worked for us
- Persistent UI label: we added a small, always-visible tag near the assistant invocation point — not a modal you can dismiss and forget.
- Lightweight modal on first use: the first time a user triggers the assistant, show a brief modal with the disclosure and a "Got it" ack (recorded).
- Inline provenance in suggestions: each suggested block had a faint "Suggested by AI" hint plus a link to more info and to the interaction log.
- Acceptance tracking: when users accept a suggestion, we log that event along with the suggestion text and the user ID. That made it straightforward to reconstruct who did what during an audit.
- SOPs that mandate human review for critical outputs (CAPA final text, regulatory submissions, MDR report drafts). The assistant helps draft, not decide.
All of these were designed around the idea of "controlled assistance" — the AI helps, humans decide, and everything is reviewable.
What auditors will likely expect to see
- Clear, discoverable disclosure in the UI or process where the AI is used.
- Training records that show users were informed and know their responsibilities.
- Traceable logs of AI interactions and whether content was accepted/modified.
- Updated risk assessments and verification activities showing you considered the AI's limitations.
- A living record of the model and provider used at the time of interactions.
Our immediate next steps (practical, not philosophical)
- Replace "probably fine" with three deliverables in the next sprint: visible disclosure on the assistant, interaction logging, and an SOP update.
- Create a one-page evidence pack for audits showing when the assistant was deployed, disclosure wording history, and example logs.
- Schedule a short legal/regulatory review and capture their signoff in the evidence pack.
If you're in a similar situation — small-ish team, Class II device, AI-assisted workflows — these are the pragmatic moves that change "probably" to "provable."
What wording did you use for disclosure in a QMS or clinical context that held up to internal or external review?
Top comments (0)