Guidance for the Development of Artificial Intelligence Reporting Guidelines in Healthcare
Long Ge, Sibo Du, Jiayi Liu, Bo Tong, Honghao Lai, Ana Marušić, Hiroaki Saito, Karen Spruyt, Akihiko Ozaki, Claudia Marcela Vélez, Yngve Falck-Ytter, Loai Albarqouni, Akira Kuriyama, Ivan D. Florez, Laura Boeira, Zachary Munn, Małgorzata M. Bała, Paweł Jemioło, Liliya Eugenevna Ziganshina, Joanne Khabsa, Shahnaz Sultan, Dena Zera, Myeong Soo Lee, Reem A. Mustafa, Janne EstillIntroduction:
The rapid proliferation of Artificial Intelligence (AI) in healthcare has been accompanied by an exponential growth in AI-specific reporting guidelines. However, the methodological rigor underpinning these guidelines varies substantially, and no standardized framework exists to guide their development. Existing frameworks derived from general health research do not adequately address the unique technical, iterative, and multidisciplinary challenges inherent to AI. To develop an evidence-based, expert-validated methodological guidance framework for the development of AI reporting guidelines in healthcare, explicitly adapted to the technical and clinical complexities of modern AI systems.
Methods:
We first conducted a scoping review to systematically map the development methodologies of 98 existing AI reporting guidelines identified from 24,268 records across five databases. The findings were synthesized into a draft framework, grounded in the WHO Handbook for Guideline Development, the EQUATOR Network standards, and the seminal 2010 framework by Moher et al. The draft framework was subsequently refined through a multidisciplinary international expert consultation. Structured deliberation during the consensus meeting resulted in targeted revisions to key steps. The final guidance framework reflects both the empirical evidence synthesis and the expert-ratified modifications.
Results:
The scoping review revealed that existing AI reporting guidelines suffer from three critical structural deficiencies: the near-universal absence of update mechanisms (99%), limited adoption of formal consensus methods (27.6%), and heterogeneous adherence to established development steps (mean completeness: 29.4%; range across individual steps: 1.0%–62.2%). Drawing on these empirical findings, we constructed an initial 20-step draft framework organized into six phases: (I) Initial Planning, (II) Pre-Meeting Preparation, (III) Consensus Development, (IV) Documentation and Publication, (V) Post-Publication Activities, and (VI) Continuous Improvement. The sixth phase represents a novel addition to conventional guideline development models, specifically addressing the sustainability crisis identified in the scoping review. Subsequent expert consultation validated the overall six-phase architecture and led to substantive refinements across several steps, primarily enhancing terminological accessibility, contextual flexibility, and operational feasibility while preserving the AI-specific innovations of the original design.
Conclusions:
This guidance provides the first expert-validated, AI-specific methodological framework for the development of reporting guidelines in healthcare, addressing a critical methodological gap. Its adoption is anticipated to improve the transparency, rigor, and sustainability of future AI guideline development efforts.