Illusion of knowledge in statistics among clinicians: evaluating the alignment between objective accuracy and subjective confidence, an online survey

Cognitive Research: Principles and Implications

卷:8

Original Article

Mehdi Khamassi¹  Benjamin Rohaut²  Marion Rouault³  François-Xavier Lejeune⁴  Camille Lakhlifi⁵ 

[1] Institute of Intelligent Systems and Robotics, CNRS, Sorbonne Université, Paris, France;

[2] Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France;AP-HP, Hôpital de la Pitié Salpêtrière, DMU Neurosciences, Paris, France;

[3] Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France;Département d’Études Cognitives, École Normale Supérieure, Université Paris Sciences & Lettres (PSL University), Paris, France;

[4] Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France;Hôpital de la Pitié Salpêtrière, Paris Brain Institute’s Data Analysis Core, Paris, France;

[5] Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, Inserm, CNRS, APHP, Hôpital de la Pitié Salpêtrière, Paris, France;Université Paris Cité, Paris, France;

关键词: Statistical illiteracy;    Metacognition;    Overconfidence bias;    Sensitivity;    Calibration;    Discrimination;    Decision-making;    Medical context;    Conditional probabilities;    Natural frequencies;   

DOI  :  10.1186/s41235-023-00474-1

 received in 2022-05-25, accepted in 2023-03-29,  发布年份 2023

来源: Springer

【 摘 要 】

Healthcare professionals’ statistical illiteracy can impair medical decision quality and compromise patient safety. Previous studies have documented clinicians’ insufficient proficiency in statistics and a tendency in overconfidence. However, an underexplored aspect is clinicians’ awareness of their lack of statistical knowledge that precludes any corrective intervention attempt. Here, we investigated physicians’, residents’ and medical students’ alignment between subjective confidence judgments and objective accuracy in basic medical statistics. We also examined how gender, profile of experience and practice of research activity affect this alignment, and the influence of problem framing (conditional probabilities, CP vs. natural frequencies, NF). Eight hundred ninety-eight clinicians completed an online survey assessing skill and confidence on three topics: vaccine efficacy, p value and diagnostic test results interpretation. Results evidenced an overall consistent poor proficiency in statistics often combined with high confidence, even in incorrect answers. We also demonstrate that despite overconfidence bias, clinicians show a degree of metacognitive sensitivity, as their confidence judgments discriminate between their correct and incorrect answers. Finally, we confirm the positive impact of the more intuitive NF framing on accuracy. Together, our results pave the way for the development of teaching recommendations and pedagogical interventions such as promoting metacognition on basic knowledge and statistical reasoning as well as the use of NF to tackle statistical illiteracy in the medical context.

【 授权许可】

CC BY   
© The Author(s) 2023

【 图 表 】

Fig. 4

Fig. 3

Fig. 3

Fig. 4

Fig. 3

Fig. 1

Fig. 1