Patient expectations for artificial intelligence transparency and agency: A qualitative study and proportionate framework for clinical artificial intelligence implementation

Artificial intelligence (AI) integration in healthcare is expanding rapidly, encompassing applications from ambient clinical scribes to diagnostic support tools. While AI adoption by physicians has recently doubled, research regarding patient expectations for transparency and agency remains limited. Current literature often fails to provide actionable guidance for translating patient perspectives into operational health system frameworks. This study utilizes feedback from Patient and Family Advisory Councils (PFACs) to explore community perspectives on healthcare AI and propose a transparency framework that engenders trust and prioritizes patient-centered care. In this qualitative study, 20 semi-structured interviews and intake surveys of PFAC members across nine hospital sites within a large healthcare network were conducted. Data analysis of interview transcripts employed a hybrid inductive and deductive thematic approach, utilizing Google’s Gemini 2.5 Pro for initial coding and theme generation, followed by rigorous human validation to reach interpretative consensus. Qualitative analysis identified four nested themes: 1) Ethical Prerequisites for AI Integration, 2) Systemic Prerequisites for Trust, 3) Preserving Clinician Oversight, and 4) Strategies for Meaningful Communication. Participants viewed transparency as an ethical “floor” necessary for autonomy and expressed that trust must be earned through evidence of systemic reliability and High Reliability Organization (HRO) principles. Critically, patients expect clinicians to serve as a “bridge” between technology and the patient, acting as the final arbiter of care decisions. These findings provide the foundation for a proportionate framework regarding communication that scales transparency based on clinical risk and the required level of patient agency. The framework categorizes AI engagement into three tiers: disclosure, assent, and consent. By nesting ethical and systemic foundations within actionable steps for clinician oversight and proactive communication, health systems can implement AI in a trust-engendering manner that preserves the sanctity of the patient-provider relationship.