There is no such thing as a validated prediction model

BMC Medicine

Opinion

Ewout W. Steyerberg¹  Ben Van Calster²  Laure Wynants³  Maarten van Smeden⁴ 

[1] Department of Development and Regeneration, KU Leuven, Leuven, Belgium;Department of Development and Regeneration, KU Leuven, Leuven, Belgium;EPI-Center, KU Leuven, Leuven, Belgium;Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands;Department of Development and Regeneration, KU Leuven, Leuven, Belgium;EPI-Center, KU Leuven, Leuven, Belgium;Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands;Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, 3584 CG, Utrecht, Netherlands;

关键词: Risk prediction models;    Predictive analytics;    Internal validation;    External validation;    Heterogeneity;    Model performance;    Calibration;    Discrimination;   

DOI  :  10.1186/s12916-023-02779-w

 received in 2022-10-13, accepted in 2023-02-10,  发布年份 2023

来源: Springer

【 摘 要 】

BackgroundClinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to claim that a prediction model works well in the intended clinical context?Main bodyWe argue to the contrary because (1) patient populations vary, (2) measurement procedures vary, and (3) populations and measurements change over time. Hence, we have to expect heterogeneity in model performance between locations and settings, and across time. It follows that prediction models are never truly validated. This does not imply that validation is not important. Rather, the current focus on developing new models should shift to a focus on more extensive, well-conducted, and well-reported validation studies of promising models.ConclusionPrincipled validation strategies are needed to understand and quantify heterogeneity, monitor performance over time, and update prediction models when appropriate. Such strategies will help to ensure that prediction models stay up-to-date and safe to support clinical decision-making.

【 授权许可】

CC BY   
© The Author(s) 2023

【 图 表 】

Fig. 4

Fig. 1