Random forests for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards

Population Health Metrics

Research

Sean Green¹  Christopher JL Murray²  Spencer L James²  Abraham D Flaxman²  Alireza Vahdatpour² 

[1] Bill & Melinda Gates Foundation, Seattle, USA;Institute for Health Metrics and Evaluation, Texas A&M University, 2301 Fifth Ave., 98121, Seattle, WA, USA;

关键词: Verbal autopsy;    cause of death certification;    validation;    machine learning;    random forests;   

DOI  :  10.1186/1478-7954-9-29

 received in 2011-04-14, accepted in 2011-08-04,  发布年份 2011

来源: Springer

【 摘 要 】

BackgroundComputer-coded verbal autopsy (CCVA) is a promising alternative to the standard approach of physician-certified verbal autopsy (PCVA), because of its high speed, low cost, and reliability. This study introduces a new CCVA technique and validates its performance using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 verbal autopsies (VAs).MethodsThe Random Forest (RF) Method from machine learning (ML) was adapted to predict cause of death by training random forests to distinguish between each pair of causes, and then combining the results through a novel ranking technique. We assessed quality of the new method at the individual level using chance-corrected concordance and at the population level using cause-specific mortality fraction (CSMF) accuracy as well as linear regression. We also compared the quality of RF to PCVA for all of these metrics. We performed this analysis separately for adult, child, and neonatal VAs. We also assessed the variation in performance with and without household recall of health care experience (HCE).ResultsFor all metrics, for all settings, RF was as good as or better than PCVA, with the exception of a nonsignificantly lower CSMF accuracy for neonates with HCE information. With HCE, the chance-corrected concordance of RF was 3.4 percentage points higher for adults, 3.2 percentage points higher for children, and 1.6 percentage points higher for neonates. The CSMF accuracy was 0.097 higher for adults, 0.097 higher for children, and 0.007 lower for neonates. Without HCE, the chance-corrected concordance of RF was 8.1 percentage points higher than PCVA for adults, 10.2 percentage points higher for children, and 5.9 percentage points higher for neonates. The CSMF accuracy was higher for RF by 0.102 for adults, 0.131 for children, and 0.025 for neonates.ConclusionsWe found that our RF Method outperformed the PCVA method in terms of chance-corrected concordance and CSMF accuracy for adult and child VA with and without HCE and for neonatal VA without HCE. It is also preferable to PCVA in terms of time and cost. Therefore, we recommend it as the technique of choice for analyzing past and current verbal autopsies.

【 授权许可】

CC BY   
© Flaxman et al; licensee BioMed Central Ltd. 2011