167 - Noise: an under recognized problem in medical decision making?

Publication Number: 167.5948

Kayla Dlugos, Hospital for Sick Children, Toronto, ON, Canada; Osami Honjo, The Hospital for Sick Children, Toronto, ON, Canada; Mjaye Mazwi, Seattle Children's, Seattle, WA, United States

Background: Variability in day-to-day decision making known as ‘noise’ negatively affects reproducibility and quality of decision making. The prevalence and significance of noise in medical decision making is unknown. A “noise audit” can characterize noise in medical decisions to assess its prevalence.

Objective: We hypothesized that noise is under-recognized and pervasive in medical decision making. We predicted that complexity of decision making is positively correlated with noise and that years of experience would mitigate the noisiness of medical decision making.

Design/Methods: A noise audit which employed low and high complexity decision scenarios was distributed to 71 Heart Center clinicians at the Hospital for Sick Children and Seattle Children’s Hospital covering common care decisions in two lesions: transposition of the great vessels and critical aortic stenosis. Variance cannot be calculated for categorical responses, so we utilize entropy and calculate it to quantify consensus level in responses.

Results: 71 (n=71) Heart Centre staff completed the survey. Practice areas included cardiac critical care (n=20), cardiology (n=33), cardiovascular surgery (n=15), heart failure and transplant (n=2), and other (n=1). Years of experience ranged from 8-23. Entropy was calculated relative to maximum possible entropy and reported for all responses. We found that noise was pervasive in medical decision making, evident through calculated aggregate entropy ratios. Overall aggregate entropy ratio calculated for the survey was 0.85, indicating lower complexity questions had smaller entropy than higher complexity questions. Entropy of response stratified by role and experience were comparable. The aggregate entropy ratio for those with >10 years of experience was 0.8, comparable to a ratio of 0.87 for those with < 10 years of experience. The aggregate entropy ratios for those in cardiac critical care, cardiology, and cardiovascular surgery were 0.85, 0.83, and 0.96 respectively.

Conclusion(s): Noise was pervasive in medical decision making in analysis of responses to common medical decisions made for patients with congenital heart disease and can be quantified to facilitate comparisons. More complex decisions were associated with noisier decision making and experience and role did not offset this effect. It is probable that this is an under-recognized problem that has significant impacts on the reproducibility, reliability, safety and efficiency of care. Further study is required to ascertain the importance of these effects.