DEV Community

Multigrid
Multigrid

Posted on • Originally published at multigrid.ai

The FDA's AI/ML SaMD Action Plan: What It Committed To

In January 2021 the FDA published a five-part action plan for AI/ML-based software as a medical device. It is still the document most often cited for what the agency intends, five years on — which makes the useful question not what it said but which parts of it have since turned into something you can actually cite.

Where the plan came from

The plan responded to comments on an April 2019 discussion paper, “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning-Based Software as a Medical Device”, which introduced a total product life cycle approach and the idea of authorising anticipated modifications in advance. The 2021 document is titled “Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan” and is published by the Center for Devices and Radiological Health. The FDA maintains a landing page for its AI/ML software as a medical device work.

It is a plan, not a rule and not a guidance document. It creates no obligation on anybody and confers no rights. Citing it as though it established a requirement is a common and consequential error — the requirements live in the statute, the regulations, and the guidance documents the plan promised.

Not legal or regulatory advice. Whether a particular product is a device at all, and which pathway applies, are determinations to make with regulatory counsel and, where appropriate, through the FDA’s own pre-submission process.

The five commitments

  • A tailored regulatory framework. Issue draft guidance on the predetermined change control plan concept from the 2019 paper, covering what a manufacturer would submit to describe anticipated modifications and the methodology for making them.
  • Good Machine Learning Practice. Support development of consensus practices for the development and evaluation of machine learning in devices, working with standards bodies and international counterparts.
  • A patient-centred approach including transparency.Develop what users of these devices need to be told — how the model was developed, on what population it was evaluated, how it performs and how it may change — beginning with a public workshop.
  • Regulatory science methods for bias and robustness.Support research into methods for evaluating and improving machine learning algorithms, including identification and elimination of bias, and evaluation of robustness and resilience to shifts in clinical inputs.
  • Real-world performance monitoring. Work with stakeholders on approaches to gathering and using real-world performance data for AI/ML devices, including through voluntary pilots.

What has been delivered

The first commitment has been fully discharged and then some. Draft guidance on predetermined change control plans came in April 2023 and final guidance in December 2024; the mechanism also acquired an express statutory footing in section 515C of the Federal Food, Drug, and Cosmetic Act via the Food and Drug Omnibus Reform Act of 2022, which is more than the plan promised. See the PCCP mechanism in detail.

The second produced a concrete artefact quickly: ten guiding principles for Good Machine Learning Practice, published in October 2021 jointly with Health Canada and the United Kingdom’s Medicines and Healthcare products Regulatory Agency. The FDA publishes the guiding principles. They are principles rather than a standard — multidisciplinary expertise across the product life cycle, representativeness of training data, independence of training and test sets, human-factors consideration of the human-AI team, and so on. Useful as a checklist for a submission narrative; not something you can certify against.

The third produced a public workshop in October 2021 and, in June 2024, a further set of guiding principles on transparency for machine learning-enabled medical devices, again with the same international partners. FDA transparency guiding principles. The agency has also continued to expand what it expects in labelling, which is where transparency actually bites.

The fourth and fifth have produced research, workshops and further principles rather than a governing document. That is not a criticism of the agency — there is no consensus method for evaluating bias in a clinical model across sites, and a guidance document asserting one would be premature — but it does mean that a manufacturer asking “what does the FDA require on bias evaluation” will find expectations expressed through review questions rather than a citable rule.

What remains open

Three gaps are worth stating plainly, because each is a place where an adviser might be tempted to fill in a confident answer.

Real-world performance monitoring has no settled mechanism. The plan contemplated pilots. There is no general requirement to instrument a deployed AI device for performance drift and report it, beyond the existing medical device reporting obligations for adverse events, and the existing obligations are a poor fit: a model quietly losing sensitivity at one site does not produce a reportable event until it produces a harm.

Generative and foundation models were not what the plan was about. The 2021 plan is written around a locked or periodically retrained classifier with a defined input and a defined output. Devices built on general-purpose language models raise questions — open-ended output, prompt-dependence, an upstream model the manufacturer does not control — that the plan does not address. The agency has convened its Digital Health Advisory Committee on generative AI in devices, and further guidance has been signalled; treat anything more specific than that as speculation until a document exists.

Clinical decision support remains the boundary fight. Whether a given software function is a device at all turns on the clinical decision support exclusion in the statute and the FDA’s September 2022 guidance interpreting it. That question sits upstream of everything in the action plan and is where most disagreements between manufacturers and the agency actually occur.

How to read the plan now

Use it as a map of the agency’s intent and a bibliography of what it has issued, not as a source of requirements. For a submission, the documents that matter are the PCCP guidance, the guidance on clinical decision support software, the software-modification guidances, the general premarket submission recommendations for device software functions, and the January 2025 draft guidance on AI-enabled device software functions across the total product life cycle — the last of which is draft, and draft guidance represents current thinking rather than a requirement.

And check dates before citing. The distance between a 2021 plan, a 2023 draft, a 2024 final and a 2025 draft is exactly the kind of detail that makes the difference between a regulatory strategy that survives review and one that gets a deficiency letter. For the wider frame, see AI medical device regulation and, for the European side of the same product, where the MDR and the AI Act overlap.

Related

Top comments (0)