245 - Intent vs. Impact: How the ACGME Annual Survey Informs NPM Fellowship Improvements

Publication Number: 245.5505

Kristen Glass, Pennsylvania State University College of Medicine, Hershey, PA, United States; Erynn Bergner, University of Oklahoma Health Sciences Center, Oklahoma City, OK, United States; Jennifer M. Brady, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, United States; Elizabeth A. Hillman, McGovern Medical School at UTHealth Houston, Houston, TX, United States; Cynthia S. Crabtree, University of Louisville - Department of Pediatrics, Louisville, KY, United States; Christie J.. Bruno, Yale-New Haven Children's Hospital, New Haven, CT, United States; Megan Gray, University of Washington School of Medicine, Seattle, WA, United States

Background: Compliance with annual Accreditation Council for Graduate Medical Education (ACGME) resident and faculty surveys is a requirement of all accredited Neonatal-Perinatal Medicine (NPM) fellowship programs. Utilizing a quality improvement methodology, program leadership invests resources in survey preparation and interpretation of survey results to inform program change.

Objective: Describe how NPM programs prepare for and utilize the annual ACGME program survey to inform program change.

Design/Methods: The Organization of Neonatal-Perinatal Training Program Directors (ONTPD) distributed an annual survey to all NPM programs in September 2024. Descriptive statistics for quantitative questions and thematic qualitative analysis of free text responses were used to summarize findings.

Results: 87 NPM fellowships contributed data on their experience with the ACGME annual program surveys, representing 81% of active NPM fellowships in 2024. The majority of programs reported preparation before the survey administration to both fellows (93%) and faculty (89%). Various preparation strategies were utilized (Table 1). The majority of programs agreed that increased transparency in the wording of survey questions was needed in order to develop effective program interventions (96%), with 88% agreeing that mathematical reporting of specialty and national comparisons is needed to adequately interpret survey results. Fellowship outcomes resulting from ACGME survey data were variable, ranging from change in program leadership and resources, to citations, site visits and probation (Table 2). NPM programs indicated ACGME survey data also lent support for program improvements such as fellow protected learning time, curricular change, procedural training and resources like increased faculty support for teaching and mentorship, and FTE for program leadership.

Conclusion(s): NPM fellowship programs utilize various strategies for preparing fellows and faculty for the annual ACGME surveys. Results of ACGME surveys can be difficult to interpret, with serious potential regulatory consequences. Increased transparency of ACGME survey questions and alignment of data reporting with quality improvement methodology is desired amongst NPM fellowship leaders. These changes may improve the robustness of NPM fellowship program quality initiatives and, ultimately, fellowship outcomes.

Table 1: Fellowship Program Preparation Prior To ACGME Surveys
*Respondents able to select more than one

Table 2: Fellowship Program Outcomes From ACGME Annual Surveys
*Respondents able to select more than one