Artificial Intelligence to Support Oral Reading Fluency Assessment in English and Spanish for Students With Reading Disabilities

Large Language Models (LLM) have the capacity to quickly produce passages of varying lengths, complexity, genres, and topics, which could be useful for teachers of mono- and multilingual students with reading-specific learning disabilities, specifically relating to monitoring oral reading fluency (ORF). This study addressed one primary research question with two sub-questions. Research Question 1: What is the quality of third-grade ORF passages generated by LLMs as compared to validated third-grade ORF in English and Spanish? Research Question 1a: What is the quantitative readability of LLM-generated passages? Research Question 1b: What is the conceptual diversity and co-occurrence of themes in said passages? Analysis was conducted using natural language processing tools, including Coh-Metrix, MultiAzterTest, and Leximancer. Results indicate that readability metrics vary greatly across texts generated by LLMs (ChatGPT, Claude), even with consistent prompting; the use of LLMs to produce ORF passages for progress monitoring and high-stakes decision-making is not recommended at this time.