Reliability of a Progressive Learning Autonomy Assessment in a Periodontology Residency Program
Ashwin Shastri, Craig Edberhardt, Ashwaq Khan, Hawra AlQallaf, GeorgeJ. Eckert, Vanchit John, Monica P. GibsonABSTRACT
Purpose
This study aimed to evaluate the reliability of a surgical progressive learning autonomy assessment tool implemented in a postgraduate periodontology residency program.
Methods
A prospective observational study was conducted during the Spring and Summer 2025 semesters using a structured, real‐time surgical assessment tool. Faculty assessed resident performance after each surgical procedure across domains of autonomy, performance, and case complexity on a five‐point scale, while residents completed parallel self‐assessments for the same procedures. Inter‐rater reliability among faculty and agreement between faculty and residents were evaluated using intraclass correlation coefficients (ICCs). Mixed‐model ANOVA was used to compare effects of resident year, procedure type, and rater group (faculty or resident) on mean scores.
Results
A total of 561 surgical assessments were analyzed. Faculty inter‐rater reliability ranged from low to moderate, with the highest agreement observed for implant placement procedures (ICC = 0.58) and lower agreement for crown lengthening (ICC = 0.13) and extraction with ridge preservation (ICC = 0.09). Agreement between faculty and residents was stronger than inter‐faculty agreement for most procedures (ICC > 0.3 for crown lengthening, guided bone regeneration, impant placement osseous). Faculty scores were consistently higher than resident self‐scores ( p < 0.05 for several procedure–question combinations by 0.3–0.5 points). Residents demonstrated significant progression in scores from Years 1 to 3 for selected procedures, particularly crown lengthening, soft tissue grafting, and extraction‐related surgeries. Graduating residents demonstrated competency based on both faculty and self‐assessment benchmarks.
Conclusions
A progressive learning autonomy assessment provides reliable, meaningful data on resident development, and highlights differences between perceived and observed competence. This model supports real‐time feedback, resident growth, and faculty calibration, with implications for broader adoption in dental specialty education.