Evaluating predictive modeling algorithms to assess patient eligibility for clinical trials from routine data

BMC Medical Informatics and Decision Making

Research Article

Hans-Ulrich Prokosch¹  Felix Köpcke¹  Dennis Toddenroth¹  Carla Nau²  Axel Scholler²  Dorota Lubgan³  Rainer Fietkau³  Roland Croner⁴  Michael Stürzl⁵ 

[1] Chair of Medical Informatics at the University Erlangen-Nuremberg, Krankenhausstraße 12, 91054, Erlangen, Germany;Department of Anesthesiology, Erlangen University Hospital, Erlangen, Germany;Department of Radiation Oncology, Erlangen University Hospital, Erlangen, Germany;Department of Surgery, Erlangen University Hospital, Erlangen, Germany;Division of Molecular and Experimental Surgery, Erlangen University Hospital, Erlangen, Germany;

关键词: Random Forest;    Electronic Health Record;    Procedure Code;    Clinical Decision Support System;    Electronic Health Record Data;   

DOI  :  10.1186/1472-6947-13-134

 received in 2013-02-13, accepted in 2013-12-02,  发布年份 2013

来源: Springer

【 摘 要 】

BackgroundThe necessity to translate eligibility criteria from free text into decision rules that are compatible with data from the electronic health record (EHR) constitutes the main challenge when developing and deploying clinical trial recruitment support systems. Recruitment decisions based on case-based reasoning, i.e. using past cases rather than explicit rules, could dispense with the need for translating eligibility criteria and could also be implemented largely independently from the terminology of the EHR’s database. We evaluated the feasibility of predictive modeling to assess the eligibility of patients for clinical trials and report on a prototype’s performance for different system configurations.MethodsThe prototype worked by using existing basic patient data of manually assessed eligible and ineligible patients to induce prediction models. Performance was measured retrospectively for three clinical trials by plotting receiver operating characteristic curves and comparing the area under the curve (ROC-AUC) for different prediction algorithms, different sizes of the learning set and different numbers and aggregation levels of the patient attributes.ResultsRandom forests were generally among the best performing models with a maximum ROC-AUC of 0.81 (CI: 0.72-0.88) for trial A, 0.96 (CI: 0.95-0.97) for trial B and 0.99 (CI: 0.98-0.99) for trial C. The full potential of this algorithm was reached after learning from approximately 200 manually screened patients (eligible and ineligible). Neither block- nor category-level aggregation of diagnosis and procedure codes influenced the algorithms’ performance substantially.ConclusionsOur results indicate that predictive modeling is a feasible approach to support patient recruitment into clinical trials. Its major advantages over the commonly applied rule-based systems are its independency from the concrete representation of eligibility criteria and EHR data and its potential for automation.

【 授权许可】

CC BY   
© Köpcke et al.; licensee BioMed Central Ltd. 2013