484 - Cash Catalyzes Culture: Improving Assessment Completion Rates

Publication Number: 484.4083

Kira R. Sieplinga, Helen DeVos Children's Hospital/Corewell Health, Grand Rapids, MI, United States; Jason Thomas, Corewell Health Helen DeVos Children's Hospital, Grand Rapids, MI, United States; Saron Mogos, Helen DeVos Children's Hospital, grand rapids, MI, United States

Background: Delivering high quality, timely summative assessments are an imperative component of graduate medical education. The Ottawa criteria for good assessment includes having feasibility and catalytic effects. The catalytic effect of assessment provides results and feedback in a fashion that motivates all stakeholders to create, enhance, and support education.
In 2018, as a response to the ACGME annual program survey (Figure 1), our pediatric residency program sought to improve the institutional culture of feedback. We completed a modified failure mode effect analysis (FMEA) to determine factors impacting resident satisfaction with faculty feedback. Timely completion of assessments was an identified vulnerability in our single site, community based pediatric residency training program. In 2023, 2367 summative assessments were sent to faculty–only 30% of these were completed on time, 41% were never completed.

Objective: Our aim was to increase the percent completed of on time pediatric trainee summative assessments from a baseline of 30% to 50% and maintain for one calendar year.

Design/Methods: We implemented multiple interventions in response to the key drivers identified in Figure 2. To successfully employ interventions, we created a reproducible tracking tool. Faculty feedback was used to ensure feasibility of success, defined as 50% completion of assessments within 14 days of the end of rotation.

Results: Figure 3 (run chart) demonstrates a significant impact of financial incentives with immediate success upon the first month of implementation. Overall, the total compliance rate increased from a baseline of 30% in 2023 to 75% to date in 2024. Lagging measures include a 20% decrease in total number of assessments (2367 in 2023 to a projected 1900 in 2024) which represents improved accuracy of assessments assigned to faculty.

Conclusion(s): Engagement of stakeholders in education and creation of financial incentives served as a catalytic effect to enhance assessment completion rates. We believe that we will see sustained increases in the percent of pediatric residents satisfied with faculty members' feedback in the upcoming ACGME annual survey which further solidifies our improved culture of feedback.

Figure 1: Percent of Pediatric Residents Satisfied with Faculty Members' Feedback


Figure 2: Key Driver Diagram


Figure 3: Average Percent Completed on Time


Figure 1: Percent of Pediatric Residents Satisfied with Faculty Members' Feedback


Figure 2: Key Driver Diagram


Figure 3: Average Percent Completed on Time