223 - Leveraging Artificial Intelligence (AI) to Assess Strengths and Weaknesses in Pediatric Clerkships for Enhanced Educational Outcomes

Publication Number: 223.6018

Roya Samuels, Zucker Hofstra Northwell SOM, Great Neck, NY, United States; Stephen Barone, Cohen Children's Medical Center, New Hyde Park, NY, United States; Melissa Pawelczak, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, United States; Jill Leibowitz, Cohen Children's Medical Center, Hempstead, NY, United States; Stacy A. McGeechan, Cohen Children's Hospital, Levittown, NY, United States

Background: In the rapidly evolving field of medical education, innovative approaches to improving student learning outcomes are essential. Traditional evaluation methods of clerkship quality rely on qualitative feedback from students, but manually parsing through large volumes of data is inefficient and hinders educational optimization. Artificial Intelligence's (AI) ability to efficiently process qualitative data enables the extraction of meaningful themes, guiding clerkship directors in prioritizing improvements. This study demonstrates how AI can enhance MS3 clerkships, pinpointing areas for development and fostering an optimized learning environment for medical students. As demands for accountability in medical education grow, AI offers a promising pathway to elevate the quality of student training.

Objective: The objective of this study is to employ AI to analyze end-of-cycle reports from ZSOM students in their pediatric clerkship, identifying strengths and weaknesses to provide actionable insights for clerkship directors. This approach aims to enhance educational outcomes and optimize the learning experience for third-year medical students on their pediatric clerkship.

Design/Methods: Natural language processing (NLP) algorithms were trained to identify recurring sentiments in six end-of-cycle pediatric clerkship reports from 95 MS3s in the 2023-24 academic year. Key areas of strength and weakness were extracted through a systematic coding process, allowing for the quantification of feedback trends. A subset of reports were manually reviewed by educational experts to validate the AI findings and ensure they aligned with expert insights.

Results: The AI analysis revealed key themes:
1. Areas of Excellence: High autonomy levels, quality teaching from faculty and residents, and diverse clinical exposure were highlighted.
2. Areas for Improvement: Students noted challenges in balancing didactic and clinical responsibilities, suggested enhancements for the ambulatory week experience, and proposed incorporating resident evaluations alongside attending feedback.
This analysis led to an AI-generated improvement plan with actionable items for the pediatric clerkship director to implement.

Conclusion(s): In conclusion, this study highlights the transformative potential of AI in medical education, as evidenced in our pediatric clerkship. By employing NLP to analyze student feedback, key strengths and weaknesses were readily identified in our clerkship experience. These findings set the stage for future initiatives that utilize technology to enhance educational practices and maintain high standards in medical training.

Key Strengths, Weaknesses, and Recommendations for Improving the Pediatric Clerkship