Diagnostic Accuracy of Artificial Intelligence as a Medical Device for Skin and Mucosal Cancer Classification Using Clinical Images: A Systematic Review and Meta-Analysis

Purpose: To assess the diagnostic accuracy of Artificial Intelligence as a Medical Device (AIaMD) for cancer classification in clinical settings, according to the PIRT framework? Participants: adults with solid malignant neoplasms of the skin and/or mucosas; Index test: AIaMD; Comparator: conventional unassisted diagnostic methods; Reference standard: histopathology and/or clinical diagnosis; Target condition: skin and mucosal cancer. Methods: Following PRISMA-DTA guidelines, a systematic search of six electronic databases and grey literature was conducted for studies published before 30 March 2026. The protocol was registered in PROSPERO (CRD42025634932). Diagnostic performance metrics (sensitivity and specificity) were extracted or estimated from contingency tables. QUADAS-2 was used to evaluate risk of bias and applicability concerns. Meta-analysis was performed using a bivariate binomial generalized linear mixed model; heterogeneity was assessed with τ2 and 95% prediction intervals, and paired comparison and meta-regression for subgroup analysis were performed. Certainty of evidence was assessed using GRADE-DTA. Results: Among 25,080 retrieved records, 24 studies were included. Overall, 83.3% of the studies were prospective, and 87.5% were related to skin cancer. The meta-analysis included 17 studies for AIaMD and 9 for unassisted clinicians. Findings revealed a pooled sensitivity of 0.893 (95% CI 0.818–0.940, τ2: 1.32) and specificity of 0.749 (95% CI 0.586–0.862, τ2: 2.38) for AIaMD, compared to sensitivity of 0.849 (95% CI 0.751–0.913, τ2: 0.620) and specificity of 0.777 (95% CI 0.624–0.880, τ2: 1.22) for clinicians. In eight paired studies, there was no significant difference in sensitivity (p = 0.74) or specificity (p = 0.85). Twenty-three of 24 (95.8%) studies had at least one domain at high risk of bias after QUADAS-2 evaluation, while GRADE-DTA indicated overall very low certainty of evidence. Conclusions: AIaMD performance was represented by high sensitivity but lower and highly variable specificity. Although clinically relevant differences cannot be excluded, AIaMD demonstrated no statistically significant differences in paired comparisons when compared to clinicians. Nonetheless, methodological limitations, factors influencing bias and applicability, and very low certainty of evidence warrant cautious interpretation and further research before real-world implementation.