Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students

Arthritis Research & Therapy

Nicolas Vuillerme¹  Eva Gruber²  Hannah Kaletta²  Georg Schett³  Melanie Hagen³  Arnd Kleyer³  Hannah Schenker³  David Simon³  Koray Tascilar³  Anna-Maria Liphardt³  Jochen Wacker³  Axel J. Hueber³  Johannes Knitza⁴  Martin Krusche⁵ 

[1]AGEIS, Université Grenoble Alpes, Grenoble, France

[2]Institut Universitaire de France, Paris, France

[3]LabCom Telecom4Health, University of Grenoble Alpes & Orange Labs, Grenoble, France

[4]Department of Internal Medicine 3 - Rheumatology and Immunology, Friedrich-Alexander University (FAU) Erlangen-Nürnberg and Universitätsklinikum Erlangen, Ulmenweg 18, 91054, Erlangen, Germany

[5]Department of Internal Medicine 3 - Rheumatology and Immunology, Friedrich-Alexander University (FAU) Erlangen-Nürnberg and Universitätsklinikum Erlangen, Ulmenweg 18, 91054, Erlangen, Germany

[6]Deutsches Zentrum für Immuntherapie (DZI), Friedrich-Alexander-University Erlangen-Nürnberg and Universitätsklinikum Erlangen, Erlangen, Germany

[7]Department of Internal Medicine 3 - Rheumatology and Immunology, Friedrich-Alexander University (FAU) Erlangen-Nürnberg and Universitätsklinikum Erlangen, Ulmenweg 18, 91054, Erlangen, Germany

[8]Deutsches Zentrum für Immuntherapie (DZI), Friedrich-Alexander-University Erlangen-Nürnberg and Universitätsklinikum Erlangen, Erlangen, Germany

[9]AGEIS, Université Grenoble Alpes, Grenoble, France

[10]Medical Department, Division of Rheumatology and Clinical Immunology, Charité-Universitätsmedizin Berlin, Berlin, Germany

关键词: Clinical decision support system;    Diagnosis;    eHealth;    Accuracy;    Apps;   

DOI  :  10.1186/s13075-021-02616-6

来源: Springer

【 摘 要 】

BackgroundAn increasing number of diagnostic decision support systems (DDSS) exist to support patients and physicians in establishing the correct diagnosis as early as possible. However, little evidence exists that supports the effectiveness of these DDSS. The objectives were to compare the diagnostic accuracy of medical students, with and without the use of a DDSS, and the diagnostic accuracy of the DDSS system itself, regarding the typical rheumatic diseases and to analyze the user experience.MethodsA total of 102 medical students were openly recruited from a university hospital and randomized (unblinded) to a control group (CG) and an intervention group (IG) that used a DDSS (Ada – Your Health Guide) to create an ordered diagnostic hypotheses list for three rheumatic case vignettes. Diagnostic accuracy, measured as the presence of the correct diagnosis first or at all on the hypothesis list, was the main outcome measure and evaluated for CG, IG, and DDSS.ResultsThe correct diagnosis was ranked first (or was present at all) in CG, IG, and DDSS in 37% (40%), 47% (55%), and 29% (43%) for the first case; 87% (94%), 84% (100%), and 51% (98%) in the second case; and 35% (59%), 20% (51%), and 4% (51%) in the third case, respectively. No significant benefit of using the DDDS could be observed. In a substantial number of situations, the mean probabilities reported by the DDSS for incorrect diagnoses were actually higher than for correct diagnoses, and students accepted false DDSS diagnostic suggestions. DDSS symptom entry greatly varied and was often incomplete or false. No significant correlation between the number of symptoms extracted and diagnostic accuracy was seen. It took on average 7 min longer to solve a case using the DDSS. In IG, 61% of students compared to 90% in CG stated that they could imagine using the DDSS in their future clinical work life.ConclusionsThe diagnostic accuracy of medical students was superior to the DDSS, and its usage did not significantly improve students’ diagnostic accuracy. DDSS usage was time-consuming and may be misleading due to prompting wrong diagnoses and probabilities.Trial registrationDRKS.de, DRKS00024433. Retrospectively registered on February 5, 2021.

【 授权许可】

CC BY