Comparison of State-of-the-Art Neural Network Survival Models with the Pooled Cohort Equations for Cardiovascular Disease Risk Prediction

BMC Medical Research Methodology

Research

Abel Kho¹  Yizhen Zhong¹  Yu Deng¹  Lihui Zhao²  Yun Zhao³  Yifan Peng⁴  Hongyan Ning⁵  Jingzhi Yu⁵  John T. Wilkins⁵  Xiaoyun Yang⁵  Norrina B. Allen⁵  Kiang Liu⁵  Donald M. Lloyd-Jones⁵  Yishu Wei⁶  Hongmei Jiang⁶  Zhiyang Zhou⁷  Lei Liu⁸  Zhiyong Lu⁹ 

[1] Center for Health Information Partnerships, Northwestern University Feinberg School of Medicine, Chicago, IL, USA;Center for Health Information Partnerships, Northwestern University Feinberg School of Medicine, Chicago, IL, USA;Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA;Department of Computer Science, University of California, Santa Barbara, CA, USA;Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA;Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA;Department of Statistics and Data Science, Northwestern University, Chicago, IL, USA;Department of Statistics, University of Manitoba, Winnipeg, MB, Canada;Division of Biostatistics, Washington University in St. Louis, St. Louis, MO, USA;National Center for Biotechnology Information, National Library of Medicine, National Institute of Health, Bethesda, MD, USA;

关键词: Artificial intelligence;    Cardiovascular disease;    Cox regression;    Deep learning;    Machine learning;    Neural network;    Pooled Cohort Equations;    Predictive modeling;    Survival analysis;   

DOI  :  10.1186/s12874-022-01829-w

 received in 2022-09-15, accepted in 2022-12-23,  发布年份 2022

来源: Springer

【 摘 要 】

BackgroundThe Pooled Cohort Equations (PCEs) are race- and sex-specific Cox proportional hazards (PH)-based models used for 10-year atherosclerotic cardiovascular disease (ASCVD) risk prediction with acceptable discrimination. In recent years, neural network models have gained increasing popularity with their success in image recognition and text classification. Various survival neural network models have been proposed by combining survival analysis and neural network architecture to take advantage of the strengths from both. However, the performance of these survival neural network models compared to each other and to PCEs in ASCVD prediction is unknown.MethodsIn this study, we used 6 cohorts from the Lifetime Risk Pooling Project (with 5 cohorts as training/internal validation and one cohort as external validation) and compared the performance of the PCEs in 10-year ASCVD risk prediction with an all two-way interactions Cox PH model (Cox PH-TWI) and three state-of-the-art neural network survival models including Nnet-survival, Deepsurv, and Cox-nnet. For all the models, we used the same 7 covariates as used in the PCEs. We fitted each of the aforementioned models in white females, white males, black females, and black males, respectively. We evaluated models’ internal and external discrimination power and calibration.ResultsThe training/internal validation sample comprised 23216 individuals. The average age at baseline was 57.8 years old (SD = 9.6); 16% developed ASCVD during average follow-up of 10.50 (SD = 3.02) years. Based on 10 × 10 cross-validation, the method that had the highest C-statistics was Deepsurv (0.7371) for white males, Deepsurv and Cox PH-TWI (0.7972) for white females, PCE (0.6981) for black males, and Deepsurv (0.7886) for black females. In the external validation dataset, Deepsurv (0.7032), Cox-nnet (0.7282), PCE (0.6811), and Deepsurv (0.7316) had the highest C-statistics for white male, white female, black male, and black female population, respectively. Calibration plots showed that in 10 × 10 validation, all models had good calibration in all race and sex groups. In external validation, all models overestimated the risk for 10-year ASCVD.ConclusionsWe demonstrated the use of the state-of-the-art neural network survival models in ASCVD risk prediction. Neural network survival models had similar if not superior discrimination and calibration compared to PCEs.

【 授权许可】

CC BY   
© The Author(s) 2023

【 图 表 】

12888_2022_4443_Article_IEq4.gif

12888_2022_4443_Article_IEq2.gif

41116_2022_35_Article_IEq398.gif

41116_2022_35_Article_IEq396.gif

41116_2022_35_Article_IEq394.gif

Fig. 3

Fig. 3

Fig. 2

Fig. 1

41116_2022_35_Article_IEq367.gif

41116_2022_35_Article_IEq362.gif

41116_2022_35_Article_IEq350.gif

41116_2022_35_Article_IEq349.gif

41116_2022_35_Article_IEq348.gif

41116_2022_35_Article_IEq346.gif

41116_2022_35_Article_IEq344.gif

41116_2022_35_Article_IEq342.gif

41116_2022_35_Article_IEq319.gif

41116_2022_35_Article_IEq318.gif

41116_2022_35_Article_IEq316.gif

41116_2022_35_Article_IEq309.gif

41116_2022_35_Article_IEq307.gif

41116_2022_35_Article_IEq306.gif

41116_2022_35_Article_IEq304.gif

41116_2022_35_Article_IEq303.gif

41116_2022_35_Article_IEq296.gif

41116_2022_35_Article_IEq294.gif

41116_2022_35_Article_IEq269.gif

41116_2022_35_Article_IEq258.gif

41116_2022_35_Article_IEq257.gif

41116_2022_35_Article_IEq256.gif

41116_2022_35_Article_IEq229.gif

41116_2022_35_Article_IEq212.gif

41116_2022_35_Article_IEq208.gif

41116_2022_35_Article_IEq189.gif

41116_2022_35_Article_IEq168.gif