QAI/ML-SaMD: A Hybrid Health–Technology Quantifiable Quality Metric for Artificial Intelligence/Machine Learning-Based Software as a Medical Device

Background: The increasing integration of Artificial Intelligence (AI) and Machine Learning (ML) into medical devices necessitates robust quality evaluation methods. However, existing approaches remain qualitative, sector-specific, or focused on isolated attributes, leaving a gap in quantifiable assessment for AI/ML-driven Software as a Medical Device (SaMD). Objective: This study introduces QAI/ML-SaMD, a novel hybrid metric that provides a comprehensive, quantifiable measure of AI/ML-SaMD quality by synthesizing health and information technology (IT) dimensions into a single composite, benchmark-ready score. Methods: The metric integrates key attributes from a systematic literature review, classified into Health and IT domains. Sub-metrics (QHealth and QIT) use weighted sums, while the overall score employs a Weighted Geometric Mean with configurable parameters to penalize domain imbalances. Validation included (a) theoretical validation against four mathematical properties, (b) an illustrative example with sensitivity analysis, (c) expert-based validation with six specialists, and (d) an evidence-based case study on FDA-authorized IDx-DR using public regulatory and clinical documentation. Results: The illustrative example yielded a score of 29.7 (“Unsuitable”). Sensitivity analysis confirmed robustness across weight, score, and combined uncertainty perturbations, with classification unchanged. Expert validation showed 83.3% agreement. The IDx-DR case study produced a score of 82.3 (“Admissible”), correctly aligning with the device’s regulatory status and supporting external validity. Conclusions: The QAI/ML-SaMD metric provides a foundational, quantifiable framework for AI/ML-SaMD quality assessment, bridging qualitative regulatory principles and measurable outcomes. It offers a practical tool for developers, regulators, and clinicians to benchmark and track quality across the SaMD lifecycle.