Direct Observation Teaching Shifts (DOTS): An Approach to Using 360-degree Assessments

Objective

Multisource feedback or 360-degree assessments are known to be a reliable and feasible evaluation tool that can have profound effects on residents. Direct observation of resident’s patient encounters is a useful educational instrument that provides information about a resident’s strengths and weaknesses but can be costly and difficult to implement on a regular basis. In order to facilitate the 360-degree assessments and enhance resident education, direct observation teaching shifts (DOTS) were created. DOTS were defined as scheduled shifts in which paired faculty/residents were assigned a chief complaint based patient encounter and the 360-degree assessment was used as the evaluation tool. The purpose of this innovation was to assess the resident’s perceptions and number of timely completed 360 evaluations after the introduction of DOTS.

Methods

Using the American Board of Emergency Medicine Model of Practice of 2016, signs, symptoms and presentations that were identified as “critical” were matched to reference textbook chapters for reading and studying purposes. Each 4 week block was assigned two distinct topics. Over a 12-week period, residents that were scheduled to the EM rotation were assigned DOTS paired with a designated faculty member, who received faculty development on DOTS and feedback. All residents had the opportunity to have at least 1 DOTS. At the completion of the 12-week period, residents and faculty were surveyed using Microsoft Forms and data was extracted and analyzed using Microsoft Excel.

Results

After completing the 12-week period, the number of 360 evaluations more than tripled per week. Most of the residents that were scheduled a DOTS completed a 360 evaluation (67%) and felt that they received individualized learning (83%) from the attending. In conclusion, the majority (75%) said that they would like to continue to be scheduled DOTS.

All the attendings involved in DOTS believe that we should continue completing the DOTS and involve more attendings.

Conclusion

With the implementation of DOTS, there was a three-fold increase in 360-degree assessments completed, and the resident’s perceived that it was educationally beneficial to them.