Procedural Skill Workplace-based Assessment of Ultrasound-guided Central Venous Catheterization: A Generalizability Study
Victor M. Neira, Ahmed Al Faraj, Sean Mackinnon, Nathan Doggett, Kaitlin Duncan, Robyn Doucet, Mullein ThorleifsonBackground:
Ultrasound-guided right internal jugular venous catheterization assessment (USG-RIJVC) poses challenges for establishing evidence-based criteria for safe and independent practice. The authors evaluated validity evidence for a Global Rating Scale (GRS) and checklist for workplace-based assessment of USG-RIJVC and established a competency standard.
Methods:
Thirty-two anesthesiology/surgery participants with different levels of experience performed USG-RIJVC in cardiac surgery patients. Their performances were independently assessed by two trained, blinded raters using a 9-item GRS and a 19-item checklist adapted from previous publications. Generalizability theory examined the internal structure. The relationship with procedural experience and between assessment instruments were evaluated using known-groups comparisons and Spearman correlation. Competency standards were established using contrasting-groups and borderline regression methods and interpreted according to the Dreyfus model of skill acquisition and the Ottawa Surgical Competency Operating Room Evaluation.
Results:
The GRS demonstrated acceptable dependability (generalizability coefficient,
Conclusion:
The GRS demonstrated stronger validity evidence than the checklist for workplace-based assessment of USG-RIJVC, providing acceptable dependability and greater precision for competency decisions. The convergence of internal structure, relationships with procedural experience, and competency standard-setting methods supports a defensible passing standard of 75%, corresponding to the transition from competent to proficient performance and readiness for indirect supervision.