AbstractFormative assessment is essential for improving teaching and learning, and AI and visualization techniques provide great potential for its design and delivery. Using NLP, cognitive diagnostic and visualization techniques designed to analyse and present students' monthly exam data, we developed an AI‐enabled visual report tool comprising six modules and conducted an empirical study of its effectiveness in a high school biology classroom. A total of 125 students in a ninth‐grade biology course were assigned to a treatment group (n = 63) receiving AI‐enabled visual reports as the intervention and a control group (n = 62) receiving overall oral feedback from the teacher. We present the main statistical results of the within‐subjects design and the between‐subjects design respectively, to better capture the main findings. Repeated measures ANOVA revealed a significant interaction effect of intervention and time on learning achievement, and the paired‐sample Wilcoxon test indicated that the treatment group had experienced increasing learning anxiety (Cohen's d = 0.203, p = 0.046) and self‐efficacy (Cohen's d = 1.793, p = 0.000) over time. Moreover, we conducted a series of non‐parametric tests to compare the effects of AI‐enabled visual reports and teacher feedback, but found no significant differences except for an increased self‐efficacy (Cohen's d = 0.312, p = 0.046). Additionally, we had the students in the treatment group rate their favourable modules in the AI‐enabled visual report and provide evaluative feedback. The study results provide important insights into the design and implementation of effective formative assessment supported by artificial AI and visualization techniques.
Practitioner notesWhat is already known about this topic
Formative assessment is essential for improving teaching and learning.
Traditional formative assessment tools lack accurate data‐oriented assessment and usability.
AI and visualization techniques have great potential for formative assessment.
What this paper adds
This study designs and implements an AI‐enabled visual report tool that generates data‐driven, user‐friendly reports.
The AI‐enabled visual report can not only enhance students' learning achievement and self‐regulated learning over time but also increase their test anxiety.
The AI‐enabled visual report has a comparable effect with teacher feedback but leads to increased self‐efficacy.
Implications for practice and/or policy
We recommend using the AI‐enabled visual report in large‐size classes for its overall positive effects on both learning achievement and self‐regulated learning.
We recommend using the AI‐enabled visual report over teacher feedback for its capacity to enhance students' self‐efficacy.
We recommend prioritizing the modules of Performance Ranking, Personal Mastery and Knowledge Alert when designing the AI‐enabled visual report.
Need a simple solution for managing your BibTeX entries? Explore CiteDrive!
- Web-based, modern reference management
- Collaborate and share with fellow researchers
- Integration with Overleaf
- Comprehensive BibTeX/BibLaTeX support
- Save articles and websites directly from your browser
- Search for new articles from a database of tens of millions of references
Try out CiteDrive More from our Archive
-
DOI: 10.1111/medu.15202 2024
Virtual education strategies in the context of sustainable health care and medical education: A topic modelling analysis of four decades of research
Jihyun Lee, Hyeongjo Kim, Frederick Kron
-
DOI: 10.1515/opar-2022-0335 2024
Identification of Ceramic Traditions on the Prehistoric Mines of Gavà (Barcelona, Spain)
Silvia Calvo Peña
-
DOI: 10.1111/medu.15254 2024
When I say … technology
Priyanka Nageswaran
-
DOI: 10.1111/medu.15258 2024
When I say … sustainability
Alexandre Medeiros Figueiredo
-
DOI: 10.1111/bjep.12651 2024
Learning from errors versus explicit instruction in preparation for a test that counts
Janet Metcalfe, Judy Xu, Matti Vuorre, Robert Siegler, Dylan Wiliam, Robert A. Bjork
-
DOI: 10.1111/medu.15309 2024
A meta‐study analysing the discourses of discourse analysis in health professions education
Anna MacLeod, Rachel H. Ellaway, Jennifer Cleland
-
DOI: 10.1177/17454999241226773 2024
Exploring the supervisors’ writing experiences and their effects on undergraduate thesis supervisory practices: A comparison of Japanese and Swedish contexts
Naoki Fujimoto-Adamson, John L Adamson, Mariya Aida Niendorf
-
DOI: 10.1002/pits.23139 2024
A preliminary comparison study of teachers' resilience in Greece and Poland
Magdalena Boczkowska, Athena Daniilidou, Maria Platsidou
-
DOI: 10.1002/pits.23136 2024
Deepening class insight through participatory lesson study: An experience with students specializing in school psychology
Diana Pasmanik, Carolina Jorquera
-
DOI: 10.5406/21627223.237.04 2024
Enculturated Pedagogy: A Dialectic Approach to Music Teaching and Learning
Benjamin C. Helton