Designing Clinician-in-the-Loop Multimodal AI for Emergency Mental and Behavioral Health Care: A Pilot Study of Consultation, Documentation, and Accountability

Emergency Department (ED) visits for mental and behavioral health (MBH) conditions have increased, intensifying demands on psychiatric consultation, documentation, risk assessment, and care coordination. While AI systems integrating computer vision and Large Language Models (LLMs) could support ED MBH care, safe implementation requires domain-specific evaluation. This preliminary study conducted 17 scenario-grounded, semi-structured interviews regarding barriers, facilitators, and design requirements for AI-enabled consultation and documentation. Thematic analysis of 277 quotations identified 18 themes. Clinicians viewed AI as a beneficial clinician-in-the-loop support tool to reduce note-taking burden, improve information synthesis, detect behavioral cues, and enhance direct patient attention. Adoption depended on workflow fit, patient acceptability, explainability, and accountability. Key requirements included clinician review of AI outputs, ligature-safe hardware, standardized kiosk placement, reliable audio/ video capture, and fallback workflows for high-risk patients. Findings suggest AI systems be evaluated as embedded sociotechnical systems within ED MBH workflows, prioritizing workload, trust, safety, and legal accountability.