The FDA has published a discussion paper proposing a competency-based framework for evaluating generative AI medical devices, modeled on clinician credentialing. The paper links device tasks to the severity of potential harm and considers premarket benchmarks plus enhanced postmarket monitoring to manage evolving systems. It asks stakeholders whether device performance should be compared to clinician panels or a median practitioner and is soliciting feedback while clarifying the effort is exploratory, not formal guidance.
FDA Eyes Doctor-Style Competency Model For Generative AI Medical Devices
Illustration: Gabriella Turrisi/Axios
The Food and Drug Administration is exploring a competency-based approach to evaluating generative AI–enabled medical devices, akin to how clinicians are assessed and credentialed, the agency says in a discussion paper first shared with Axios. The paper is exploratory and is not formal guidance for manufacturers; the FDA is soliciting stakeholder feedback as it considers new premarket and postmarket practices for these evolving tools.
Why It Matters
AI tools are moving rapidly into clinical care, from information aides to systems that provide direct patient care. Generative AI devices differ from traditional medical devices because their outputs can vary and the systems may change over time—creating new challenges for premarket evaluation and increasing the need for robust postmarket surveillance to protect patients while enabling innovation.
What The FDA Is Proposing
The discussion paper outlines a risk framework that links the type of activity a GenAI device performs to the potential severity of harm from incorrect outputs. It also introduces a competency-based pathway "inspired, at a high level, by how human clinicians are evaluated and credentialed," which would set benchmarks and require evidence of real-world performance in clinical settings.
Premarket: Competency-Based Evaluation
Under the proposed approach, the FDA would assess whether a device can consistently meet defined performance benchmarks applicable to its intended clinical tasks. Evaluation could involve comparisons against clinician performance, defined clinical scenarios, and demonstration of safe behavior across a range of inputs and edge cases.
Postmarket: Stronger Real-World Monitoring
The paper weighs whether greater reliance on postmarket monitoring could offset some premarket uncertainty for certain devices. That might include continuous performance monitoring, reporting requirements, real-world evidence collection, and mechanisms to detect and mitigate errors as models evolve or are updated.
Key Open Question: What Is The Right Standard?
One central issue the FDA raises is the appropriate comparator for device performance. The agency suggests options such as comparing AI outputs to a panel of qualified clinicians whose consensus reflects the standard of care, or to the performance of a median clinician in practice. Each choice has implications for fairness, safety, and innovation.
"We are launching a transparent process to inform the development of an approach that safeguards patients and consumers, advances innovation and serves as a potential model for regulators around the world," said Michelle Tarver, director of the FDA Center for Devices and Radiological Health.
"Artificial intelligence is transforming medicine, and the United States must lead in shaping how this technology is developed and used safely and responsibly," said Acting FDA Commissioner Kyle Diamantas.
Implications
If adopted, a competency-based pathway could allow some AI-enabled devices to reach the market with defined performance benchmarks tied to clinical outcomes, while relying on strong postmarket systems to manage uncertainty and learning. Device makers, clinicians, hospitals, and patient groups will likely play key roles in defining benchmarks, monitoring systems, and acceptable risk thresholds.
Next Steps
The FDA is in an early, consultative phase and has asked stakeholders to comment on the discussion paper. The agency emphasizes its goal of a "nimble regulatory approach that employs least burdensome principles and allows patients timely access to safe and effective medical devices." Interested parties should review the paper and provide feedback during the agency's comment period.
Help us improve.
