Preclinical Dialogue Simulation: Evaluating Response Accessibility in Conversational Artificial Intelligence for Aphasia Therapy
Gerald C. Imaezue, Krishna V. Maram, David Ajayi, Isra Alohali, Rajesh K. ButtaPurpose:
Large language model (LLM)–driven conversational agents are increasingly considered for use in clinical contexts, yet systematic approaches for evaluating their behavior in impairment-rich, speech-based therapeutic interactions remain limited. This study extends the Agent-Based Conversational Dialogue (ABCD) simulation method as a preclinical testbed to evaluate how LLMs generate accessible clinician language when responding to characteristic aphasic speech during Response Elaboration Training.
Method:
ABCD was used to simulate multi-turn spoken therapeutic dialogues between an LLM-driven clinician and an artificial intelligence (AI)–simulated aphasic patient, enabling controlled manipulation of impairment profiles, prompting strategies, and reasoning modes without human participants. Three LLM families (Claude, GPT, and Gemini) were benchmarked under zero-shot and few-shot prompting and standard versus advanced reasoning. Response accessibility was quantified using established readability metrics (Flesch Reading Ease, Dale–Chall) and a composite score derived from 16 standardized readability measures.
Results:
Distinct accessibility signatures emerged across model architectures and configurations. Few-shot prompting and advanced reasoning generally yielded more accessible clinician responses, whereas Gemini demonstrated superior accessibility under zero-shot, standard reasoning.
Conclusions:
LLMs differ systematically in their capacity to adapt clinician language to impaired speech. ABCD provides a scalable, preclinical dialogue simulation framework for benchmarking conversational AI in clinically oriented, impairment-rich dialogue across multidimensional constraints. It offers guidance for model selection and configuration prior to clinical translation in communication rehabilitation.
Supplemental Material: