Performance of four computer-coded verbal autopsy methods for cause of death assignment compared with physician coding on 24,000 deaths in low- and middle-income countries

BMC Medicine

Prabhat Jha²  Faujdar Ram⁷  Rajesh Kumar³  Abhishek Singh⁷  Suresh Kumar Rathi²  Dewan Alam⁶  Paul Mee⁴  Stephen Tollman⁵  Peter Byass¹  Jordana Leitao²  Ying Lu⁸  Pierre Miasnikof²  Lukasz Aleksandrowicz²  Nikita Desai² 

[1] Umeå Centre for Global Health Research, Division of Epidemiology and Global Health, Department of Public Health and Clinical Medicine, Umeå University, Umeå, Sweden;Centre for Global Heath Research, St. Michael’s Hospital, Dalla Lana School of Public Health, University of Toronto, Toronto Ontario, Canada;School of Public Health, Post Graduate Institute of Medical Research and Education, Chandigarh, India;Medical Research Council/Wits University Rural Public Health and Health Transitions Research Unit (Agincourt), School of Public Health, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa;International Network for the Demographic Evaluation of Populations and Their Health (INDEPTH) Network, Accra, Ghana;International Centre for Diarrhoeal Disease Research, Bangladesh (ICDDR,B), Dhaka, Bangladesh;International Institute for Population Sciences, Mumbai, Maharashtra, India;Center for the Promotion of Research Involving Innovative Statistical Methodology, Steinhardt School of Culture, Education and Human Development, New York University, New York NY, USA

关键词: Verbal autopsy;    Validation;    Tariff method;    Random forest;    Physician-certified verbal autopsy (PCVA);    King-Lu;    InterVA-4;    Computer-coded verbal autopsy (CCVA);    Causes of death;   

Others  :  855019
DOI  :  10.1186/1741-7015-12-20

 received in 2013-09-02, accepted in 2013-11-01,  发布年份 2014

【 摘 要 】

Background

Physician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (CODs) in countries where medical certification of death is uncommon. Computer-coded verbal autopsy (CCVA) methods have been proposed as a faster and cheaper alternative to PCVA, though they have not been widely compared to PCVA or to each other.

Methods

We compared the performance of open-source random forest, open-source tariff method, InterVA-4, and the King-Lu method to PCVA on five datasets comprising over 24,000 verbal autopsies from low- and middle-income countries. Metrics to assess performance were positive predictive value and partial chance-corrected concordance at the individual level, and cause-specific mortality fraction accuracy and cause-specific mortality fraction error at the population level.

Results

The positive predictive value for the most probable COD predicted by the four CCVA methods averaged about 43% to 44% across the datasets. The average positive predictive value improved for the top three most probable CODs, with greater improvements for open-source random forest (69%) and open-source tariff method (68%) than for InterVA-4 (62%). The average partial chance-corrected concordance for the most probable COD predicted by the open-source random forest, open-source tariff method and InterVA-4 were 41%, 40% and 41%, respectively, with better results for the top three most probable CODs. Performance generally improved with larger datasets. At the population level, the King-Lu method had the highest average cause-specific mortality fraction accuracy across all five datasets (91%), followed by InterVA-4 (72% across three datasets), open-source random forest (71%) and open-source tariff method (54%).

Conclusions

On an individual level, no single method was able to replicate the physician assignment of COD more than about half the time. At the population level, the King-Lu method was the best method to estimate cause-specific mortality fractions, though it does not assign individual CODs. Future testing should focus on combining different computer-coded verbal autopsy tools, paired with PCVA strengths. This includes using open-source tools applied to larger and varied datasets (especially those including a random sample of deaths drawn from the population), so as to establish the performance for age- and sex-specific CODs.

【 授权许可】

   
2014 Desai et al.; licensee BioMed Central Ltd.

【 图 表 】

【 参考文献 】

• [1]Jha P: Counting the dead is one of the world’s best investments to reduce premature mortality. Hypothesis 2012, 10:e3.
• [2]Setel PW, Sankoh O, Rao C, Velkoff VA, Mathers C, Gonghuan Y, Hemed Y, Jha P, Lopez AD: Sample registration of vital events with verbal autopsy: a renewed commitment to measuring and monitoring vital statistics. Bull World Health Organ 2005, 83:611-617.
• [3]Fottrell E, Byass P: Verbal autopsy: methods in transition. Epidemiol Rev 2010, 32:38-55.
• [4]Health and Demographic Surveillance Unit: Health and Demographic Surveillance System: Matlab. Volume 36: Registration of Health and Demographic Events 2003. Dhaka: ICDDR, B; 2005.
• [5]King G, Lu Y: Verbal autopsy methods with multiple causes of death. Statistical Science 2008, 23:78-91.
• [6]Kahn K, Collinson MA, Gómez-Olivé FX, Mokoena O, Twine R, Mee P, Afolabi SA, Clark BD, Kabudula CW, Khosa A, Khoza S, Shabangu MG, Silaule B, Tibane JB, Wagner RG, Garenne ML, Clark SJ, Tollman SM: Profile: Agincourt health and socio-demographic surveillance system. Int J Epidemiol 2012, 41:988-1001.
• [7]Murray CJ, Lopez AD, Black R, Ahuja R, Ali SM, Baqui A, Dandona L, Dantzer E, Das V, Dhingra U, Dutta A, Fawzi W, Flaxman AD, Gómez S, Hernández B, Joshi R, Kalter H, Kumar A, Kumar V, Lozano R, Lucero M, Mehta S, Neal B, Ohno SL, Prasad R, Praveen D, Premji Z, Ramírez-Villalobos D, Remolador H, Riley I, et al.: Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets. Popul Health Metr 2011, 9:27. BioMed Central Full Text
• [8]Green ST, Flaxman AD: Machine learning methods for verbal autopsy in developing countries. In AAAI Spring Symposium on Artificial Intelligence for Development (AI-D): March 22-24 2010; Stanford. Stanford: Stanford University; 2010. [https://github.com/aflaxman/verbal-autopsy-challenge webcite]
• [9]Jha P, Gajalakshmi V, Gupta PC, Kumar R, Mony P, Dhingra N, Peto R: Prospective study of one million deaths in India: rationale, design, and validation results. PLoS Med 2006, 3:e18.
• [10]Bassani DG, Kumar R, Awasthi S, Morris SK, Paul VK, Shet A, Ram U, Gaffey MF, Black RE, Jha P, Million Death Study Collaborators: Causes of neonatal and child mortality in India: a nationally representative mortality survey. Lancet 2010, 376:1853-1860.
• [11]Byass P, Chandramohan D, Clark SJ, D'Ambruoso L, Fottrell E, Graham WJ, Herbst AJ, Hodgson A, Hounton S, Kahn K, Krishnan A, Leitao J, Odhiambo F, Sankoh OA, Tollman SM: Strengthening standardised interpretation of verbal autopsy data: the new InterVA-4 tool. Glob Health Action 2012, 5:1-8. Available for download at http://www.interva.net webcite
• [12]Flaxman A, Vahdatpour A, Green S, James S, Murray C, for the Population Health Metrics Research Consortium (PHMRC): Random forests for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards. Popul Health Metr 2011, 9:29. BioMed Central Full Text
• [13]James S, Flaxman A, Murray C, and the Public Health Metrics Research Consortium: Performance of the tariff method: Validation of a simple additive algorithm for analysis of verbal autopsies. Popul Health Metr 2011, 9:31. BioMed Central Full Text
• [14]King G, Lu Y, Shibuya K: Designing verbal autopsy studies. Popul Health Metr 2010, 8:19. Available for download at http://gking.harvard.edu/va webcite BioMed Central Full Text
• [15]Murray C, Lozano R, Flaxman AD, Vadhatpour A, Lopez AD: Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies. Popul Health Metr 2011, 9:28. BioMed Central Full Text
• [16]Leitao JC, Chandramohan D, Byass P, Jakob R, Bundhamcharoen K, Choprapowan C, de Savigny D, Fottrell E, França E, Frøen F, Gewaifel G, Hodgson A, Hounton S, Kahn K, Krishnan A, Kumar V, Masanja H, Nichols E, Notzon F, Rasooly MH, Sankoh O, Spiegel P, AbouZahr C, Amexo M, Kebede D, Alley WS, Marinho F, Ali M, Loyola E, Chikersal J, et al.: Revising the WHO verbal autopsy instrument to facilitate routine cause-of-death monitoring. Glob Health Action 2013, 6:21518.
• [17]Leitao J, Desai N, Aleksandrowicz L, Byass P, Miasnikof P, Tollman S, Alam D, Lu Y, Rathi SK, Singh A, Suraweera W, Ram F, Jha P: Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review. BMC Med 2014, 12:22.
• [18]Flaxman A, Vahdatpour A, James S, Birnbaum J, Murray C, for the Population Health Metrics Research Consortium (PHMRC): Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards. Popul Health Metr 2011, 9:35. BioMed Central Full Text
• [19]World Health Organization (WHO): International Statistical Classifications of Diseases and Related Health Problems, Volume 1. 10th edition. Geneva, Switzerland: World Health Organization; 2008.
• [20]Byass P: Usefulness of the Population Health Metrics Research Consortium Gold Standard Verbal Autopsy Data for General Verbal Autopsy Methods. BMC Med 2014, 12:23.
• [21]Anker M: The effect of misclassification error on reported cause-specific mortality fractions from verbal autopsy. Int J Epidemiol 1997, 26:1090-1096.
• [22]Maude GH, Ross DA: The effect of different sensitivity, specificity and cause-specific mortality fractions on the estimation of differences in cause-specific mortality rates in children from studies using verbal autopsies. Int J Epidemiol 1997, 26:1097-1106.
• [23]Aleksandrowicz L, Malhotra V, Dikshit R, Gupta PC, Kumar R, Sheth J, Rathi SK, Suraweera W, Miasnikofi P, Jotkar R, Sinha D, Awasthi S, Bhatia P, Jha P: Performance criteria for verbal autopsy‒based systems to estimate national causes of death: development and application to the Indian Million Death Study. BMC Med 12:21.
• [24]Jha P: Reliable direct measurement of causes of death in low- and middle-income countries. BMC Med 2014, 12:19.