482 - And if you don’t know, now you... know?: Impact of medical student certainty on diagnostic and therapeutic accuracy in case-based learning

Publication Number: 482.3581

Emily Murphy, Johns Hopkins University School of Medicine, Baltimore, MD, United States; Khadijah Tiamiyu, Northwestern University The Feinberg School of Medicine, Chicago, IL, United States; Amit Pahwa, Johns Hopkins University School of Medicine, Baltimore, MD, United States

Background: Assessment of clinical reasoning in medical students (MS) traditionally uses cases with diagnostic certainty, whereas pediatricians frequently encounter uncertain cases that must be diagnosed clinically. Limited data suggest that MS may over-test and under-treat adults compared to physicians when facing uncertainty, but it is unknown how uncertainty impacts the care of pediatric patients.

Objective: We aimed to assess the impact of uncertainty on diagnostic and therapeutic accuracy across three cases of respiratory distress in a pediatric clerkship session.

Design/Methods: We performed a single-site prospective study of clinical MS participating in a one-hour case-based session during their pediatrics clerkship at our institution from September 2023 – September 2024. MS were assigned by academic quarter to one of 3 alternating cases and randomized into groups of 5-6 students per quarter. We developed three cases with overlapping and discriminating features of asthma, bronchiolitis, or pneumonia (Table 1). Groups generated a consensus 3-item differential diagnosis (DDx), selected diagnostic and therapeutic orders from a pre-defined list, and rated their diagnostic and therapeutic certainty on a 4-point scale. Responses were submitted via survey. Diagnostic accuracy was defined by the case diagnosis and therapeutic accuracy was defined by order appropriateness (an appropriate albuterol order would be “yes” for asthma, “no” for bronchiolitis). Chi square analysis and correlation with Pearson’s coefficient were performed. Certainty data were dichotomized in Chi square analyses. A two-tailed p-value of < 0.05 was considered significant. The study was approved by the IRB.

Results: Twenty-nine groups (asthma = 10, bronchiolitis = 11, pneumonia = 8) participated in 6 sessions. There were no significant differences in diagnostic accuracy or certainty between cases. We observed differences in frequency of diagnostic and therapeutic orders between cases, including therapeutic accuracy, but no differences in therapeutic certainty (Table 2). Diagnostic certainty did not correlate with diagnostic or therapeutic accuracy, but diagnostic accuracy (r=0.38, p=0.45) and certainty in albuterol use (r=0.38, p=0.044) correlated with albuterol therapeutic accuracy (Table 3). Neither diagnostic accuracy nor certainty in antibiotic use correlated with antibiotic therapeutic accuracy.

Conclusion(s): MS diagnostic and therapeutic certainty is variable and does not reliably predict accuracy. We find that MS may experience more gaps with specific diagnoses or therapies than with others, with implications for appropriate antibiotic use.

Table 1: Comparison of asthma, bronchiolitis, and community-acquired pneumonia cases
Abbreviations: CAP, community-acquired pneumonia; PMH, past medical history; SpO2, oxygen saturation; WOB, work of breathing; PO intake, oral intake; UOP, urine output; I:E ratio, inspiratory:expiratory ratio; RLL, right lower lobe

Table 2: Difference in certainty, diagnostic accuracy, and therapeutic accuracy between cases
*Chi square analysis
Abbreviations: DDX, differential diagnosis; CBC, complete blood count; ESR/CRP, erythrocyte sedimentation rate/C-reactive protein; RVP, respiratory viral panel; cx, culture; CXR, chest radiograph; CT chest, computed tomography of the chest; LFNC, low-flow nasal cannula; HFNC, high-flow nasal cannula

Table 3: Correlation between certainty, diagnostic accuracy, and therapeutic accuracy
Abbreviations: DDx, differential diagnosis; r, Pearson’s correlation coefficient

Table 1: Comparison of asthma, bronchiolitis, and community-acquired pneumonia cases
Abbreviations: CAP, community-acquired pneumonia; PMH, past medical history; SpO2, oxygen saturation; WOB, work of breathing; PO intake, oral intake; UOP, urine output; I:E ratio, inspiratory:expiratory ratio; RLL, right lower lobe

Table 2: Difference in certainty, diagnostic accuracy, and therapeutic accuracy between cases
*Chi square analysis
Abbreviations: DDX, differential diagnosis; CBC, complete blood count; ESR/CRP, erythrocyte sedimentation rate/C-reactive protein; RVP, respiratory viral panel; cx, culture; CXR, chest radiograph; CT chest, computed tomography of the chest; LFNC, low-flow nasal cannula; HFNC, high-flow nasal cannula

Table 3: Correlation between certainty, diagnostic accuracy, and therapeutic accuracy
Abbreviations: DDx, differential diagnosis; r, Pearson’s correlation coefficient