Longitudinal Changes in Feedback Quality After Nationwide Implementation of a Mobile Application for Entrustable Professional Activity Data Management in Thailand
Khuansiri Narajeenron, Stephen J Kerr, Intanon Imsuwan, Cherdsak Iramaneerat, Ara Tekian, Yoon Soo Park, , Khrongwong Musikatavorn, Tanawat Tarapan, Natchaya Treesaksrisakul, Supattra Inyu, Ar-aishah Dadeh, Suthee Intharachat, Piyatida Kalayanamitra, Pornlert Pluemchitmongkhon, Panuwong Sansomranjai, Suwasan Bunyarat, Marisa Thongnok, Sutjarit Suankool, Chanipat Meelarp, Kamonphan Thirapong, Chompunut Saengpanit, Narong Jongaramrueng, Napat Rattanawongsa, Chaiyaporn Yuksen, Chanida Kemngern, Kannika Tasasungkhin, Theerapon Tangsuwanaruk, Varalee Aphinives, Nutarpa Kulkanokwan, Phummarin Saelim, Dhanu Gaysonsiri, Watchara Petchratchatanon, Alisa Yanasan, Tanyaporn Nakornchai, Suttisak WattanasansomboonAbstract
Purpose
This national study examined longitudinal changes in perceived feedback quality during phased rollout of a mobile entrustable professional activity (EPA)-based feedback application across Thai College of Emergency Physicians-accredited emergency medicine residency programs and explored institutional, contextual, and application-use variables associated with perceived feedback quality.
Method
This national, multisite prospective observational study (December 2019 to September 2020) included 135 faculty and 357 residents from accredited Thai emergency medicine residency programs. The application was implemented in three phases: notifications, feedback guide/self-reflection prompts, and dashboard. Participants completed FEED-ER across five domains (timeliness, specificity, communication, action plan, feedback culture) using 5-point Likert-type items. Scores were converted to Percent of Maximum Possible (POMP; 0-100; higher scores indicate higher perceived feedback quality). Linear mixed-effects models estimated changes in domain and total FEED-ER POMP scores over time; total-score models assessed associations with institutional, contextual, and application-use variables.
Results
Faculty-reported overall perceived feedback quality did not change significantly at 3 months but increased at 6 months (4.03 POMP; 95% CI, 1.99 to 6.07) and 9 months (3.57; 95% CI, 1.02 to 6.11). Resident-reported overall scores did not change significantly. In resident models, dashboard review (always 2.54; 95% CI, 0.18 to 4.89; sometimes 2.21; 95% CI, 0.32 to 4.09) and resident rules (1.78; 95% CI, 0.09 to 3.48) were associated with higher scores; absence of formal feedback strategies was associated with lower scores (-2.00; 95% CI, -3.87 to -0.13).
Conclusions
During implementation of a mobile EPA-based feedback application, faculty reported modest improvements in perceived feedback quality, whereas resident perceptions remained stable. Digital tools may support feedback processes when aligned with institutional context. Future studies should link feedback to encounters and evaluate mechanisms and downstream educational outcomes using designs that strengthen causal inference.