The Interrater Reliability of Cardiac Arrest Video Review: A Pilot Study

Audio and video recording of cardiac arrest (CA) is a valuable quality improvement, education, and research tool, but the interrater reliability (IRR) of data collection is not well studied. We aimed to evaluate the IRR of data abstracted from our review of video recordings of Emergency Department (ED) CAs. In this retrospective study, 2 blinded, expert physicians reviewed 21 randomly selected CA videos from a single academic ED. Predefined categorical (eg, initial rhythm) and noncategorical (eg, time to defibrillation, duration of compression interruptions) variables were abstracted. IRR was evaluated using Cohen’s kappa for categorical variables and interclass correlation coefficient (ICC) with a 2-way mixed-effects model for noncategorical variables, including nested analysis for repeated events. IRR was excellent for noncategorical and time-to-event variables, such as time to defibrillator pad attachment (ICC = 0.99), first rhythm analysis (ICC = 0.94), first shock (ICC = 1.00), number of interruptions (ICC = 0.93), and time of death (ICC = 1.00). In contrast, categorical variables demonstrated moderate reliability for emergency medical services reported rhythm (κ = 0.61), initial ED rhythm (κ = 0.69), and endotracheal tube presence on arrival (κ = 0.70); though agreement was perfect for variables like mechanical compression or vascular access presence on arrival. The mixed linear model showed good IRR for durations of chest compression interruptions (ICC = 0.85), return of spontaneous circulation (ICC = 0.79), and number of intubation attempts (ICC = 0.96). CA video review demonstrates excellent IRR for collecting noncategorical, time-based data, but only moderate IRR for categorical variables.