The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: a mixed methods study

BMC Medicine

Research Article

Alexandra Kaider¹  Fabian Peter Hammerle²  Daniel Laxar³  Mathias Maleczek³  Oliver Kimberger³  Magdalena Eitenberger⁴ 

[1] Center for Medical Data Science, Medical University of Vienna, Vienna, Austria;Department of Anaesthesia, Intensive Care Medicine and Pain Medicine, Medical University of Vienna, Vienna, Austria;Department of Anaesthesia, Intensive Care Medicine and Pain Medicine, Medical University of Vienna, Vienna, Austria;Ludwig Boltzmann Institute Digital Health and Patient Safety, Ludwig Boltzmann Gesellschaft, Vienna, Austria;Ludwig Boltzmann Institute Digital Health and Patient Safety, Ludwig Boltzmann Gesellschaft, Vienna, Austria;

关键词: Triage;    Decision process;    Clinical decision support systems;    Machine learning;    Human–computer interaction;   

DOI  :  10.1186/s12916-023-03068-2

 received in 2023-05-12, accepted in 2023-09-05,  发布年份 2023

来源: Springer

【 摘 要 】

BackgroundDuring the COVID-19 pandemic, a variety of clinical decision support systems (CDSS) were developed to aid patient triage. However, research focusing on the interaction between decision support systems and human experts is lacking.MethodsThirty-two physicians were recruited to rate the survival probability of 59 critically ill patients by means of chart review. Subsequently, one of two artificial intelligence systems advised the physician of a computed survival probability. However, only one of these systems explained the reasons behind its decision-making. In the third step, physicians reviewed the chart once again to determine the final survival probability rating. We hypothesized that an explaining system would exhibit a higher impact on the physicians’ second rating (i.e., higher weight-on-advice).ResultsThe survival probability rating given by the physician after receiving advice from the clinical decision support system was a median of 4 percentage points closer to the advice than the initial rating. Weight-on-advice was not significantly different (p = 0.115) between the two systems (with vs without explanation for its decision). Additionally, weight-on-advice showed no difference according to time of day or between board-qualified and not yet board-qualified physicians. Self-reported post-experiment overall trust was awarded a median of 4 out of 10 points. When asked after the conclusion of the experiment, overall trust was 5.5/10 (non-explaining median 4 (IQR 3.5–5.5), explaining median 7 (IQR 5.5–7.5), p = 0.007).ConclusionsAlthough overall trust in the models was low, the median (IQR) weight-on-advice was high (0.33 (0.0–0.56)) and in line with published literature on expert advice. In contrast to the hypothesis, weight-on-advice was comparable between the explaining and non-explaining systems. In 30% of cases, weight-on-advice was 0, meaning the physician did not change their rating. The median of the remaining weight-on-advice values was 50%, suggesting that physicians either dismissed the recommendation or employed a “meeting halfway” approach. Newer technologies, such as clinical reasoning systems, may be able to augment the decision process rather than simply presenting unexplained bias.

【 授权许可】

CC BY   
© BioMed Central Ltd., part of Springer Nature 2023

【 图 表 】

Fig. 7

Fig. 3

40708_2023_202_Article_IEq10.gif

40677_2023_249_Article_IEq40.gif

Fig. 4

Fig. 7