Usability Testing to Improve the User Experience of an Electronic Patient-Reported Outcome System for Chimeric Antigen Receptor T-Cell Therapies
Foram Khatsuria, Christel McMullan, Karen L. Shaw, Anita Walker, Hayley Ellis, Rebecca Lloyd, Paul Ferguson, Elin Haf Davies, Izuchika Nduka, Lavanya Palaram, John Ansell, Evelyn Chakera, Mark Chatterley, Lester Pyatt, Julie Richardson-Abraham, Ronjon Chakraverty, Olalekan Lee Aiyegbusi, Melanie J. Calvert, Sarah E. HughesPurpose
Treatment with chimeric antigen receptor T-cell (CAR-T) therapies has led to remission in patients with relapsed/refractory hematologic malignancies. However, CAR-T therapies are associated with severe toxicities. An electronic patient-reported outcome (ePRO) system for CAR-T therapies was developed on a digital electronic clinical outcome assessment platform. The system was designed to capture and monitor symptoms and side effects in CAR-T patients within routine clinical care. This study aimed to assess the usability of the ePRO system and use the findings to inform refinements that enhance user experience.
Methods
Participants were recruited from a previous study and a National Health Service center. In two rounds of online sessions, participants completed a series of tasks, semistructured interviews, cognitive debriefing, and the system usability scale (SUS). Using the Fit between Individual, Task, and Technology framework, we evaluated system usability. Sessions were audiorecorded, transcribed, and analyzed inductively and deductively to generate themes.
Results
Fifteen participants, age 18-79 years, evaluated the ePRO system. Qualitative analysis identified eight themes. For effectiveness, 82% of tasks completed error-free for the application and 70% for the clinical dashboard. For efficiency, tasks required an average 29 minutes to complete for the application and 20 minutes for the dashboard. SUS scores indicated excellent user satisfaction.
Conclusion
This study identified key usability strengths, challenges, and areas for refinement of the ePRO system. Quantitative results showed high usability and user satisfaction, supported by qualitative insights highlighting specific issues and advantages. In response, targeted improvements were made, including refining welcome text, enhancing font readability, and updating the user manual. Overall, these findings demonstrate that the ePRO system is usable and acceptable to both patients and clinicians for symptom and quality-of-life monitoring in routine CAR-T care.