WIP 28 - ChatGPT Sims: An AI Approach to Pediatric Simulation Creation

Publication Number: WIP 28.7406

Kriti Gupta, Childrens Hospital of Philadelphia, Philadelphia, PA, United States; Michael Hrdy, Children's Hospital of Philadelphia, Philadelphia, PA, United States

Background: A large collection of pediatric emergency simulations have been written by physicians across numerous platforms internationally. Cases are written with specific learning objectives in mind to meet audience-specific goals. Traditionally, the writing process for a new simulation case can take considerable time and effort. The most common problem that occurs when implementing these scenarios in real life is the need for facilitators to adapt to last-minute changes. Some examples of last-minute adjustments is the availability of a manikin that requires changing of patient age, age-appropriate vital signs, and age-specific management. Additional examples include the adjustment of a case to meet learner-specific difficulty levels, such as a case written for novice trainees but needing modification for use with advanced trainees. Artificial intelligence models like ChatGPT can serve as an efficient platform for timely case creation that could provide solutions for such unforeseen challenges.

Objective: The primary objective of the study is to evaluate a ChatGPT-based simulation writing tool to create pediatric emergency care simulations that are comparable to those written by physicians.

Design/Methods: Content experts in pediatric emergency care simulation will use an established simulation scoring checklist, the SSET (Simulation Scenario Evaluation Tool), to evaluate four different pediatric simulation topics with two versions each (one written by a physician, one written by the ChatGPT tool). Experts will be blinded to which case is from which source. Scores will be analyzed to determine the relative performance of the AI tool. Data collection starts October 23, 2024 and ends December 31, 2024. Statistical analysis will happen in late 2024/early 2025. By January of 2025 there should be an abstract reflective of the project’s completion. This study was deemed exempt from the institutional IRB.