Evaluating a Digital Game-Based Mathematics Program: Achievement Impacts and Collaborative Learning Processes

This study implemented a randomized controlled trial (RCT) to examine the impact of EMPIRES, a collaborative, story-based digital mathematics intervention, and the association between teacher promotion of interaction and student mathematics achievement through student peer instruction, a major mechanism proposed in its theory of change. Researchers randomly assigned 199 students in grades 7 and 8 from three schools in two districts to the intervention or control condition. The estimated impact on standardized mathematics achievement was positive but not statistically significant (effect size = 0.152, p = 0.178, n = 151). A separate exploratory path analysis used data from 49 intervention students with complete survey and achievement data to examine the proposed pathway linking students’ perceptions of teacher promoting interaction to mathematics achievement through peer instruction. Perceived teacher promotion of interaction was positively associated with peer instruction (standardized coefficient (β) = 0.371, p = 0.003), which, in turn, was positively associated with posttest achievement (β = 0.231, p = 0.044). Teacher promoting interaction also had a positive direct association with achievement (β = 0.212, p = 0.016). The indirect association through peer instruction was marginally significant (β = 0.086, p = 0.095), whereas the total association was significant (β = 0.298, p = 0.001). These findings provide suggestive but inconclusive evidence that the intervention may improve achievement and are consistent with the proposed collaborative-learning mechanism of the intervention. Future studies should examine how intentionally designed collaborative activities in digital mathematics programs can be implemented in authentic classrooms to support student collaboration and achievement, and under what conditions they are most effective.