Individual participant data meta-analysis of prognostic factor studies: state of the art?

BMC Medical Research Methodology

Richard D Riley¹  Willi Sauerbrei²  Ghada Abo-Zaid³ 

[1] School of Health and Populations Sciences, Public Health Building, University of Birmingham,, Edgbaston, Birmingham, B15 2TT, UK;Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Freiburg, Stefan-Meier-Strasse 26, 79104, Germany;European Centre for Environment and Human Health, Peninsula College of Medicine and Dentistry, University of Exeter Knowledge Spa Royal Cornwall Hospital Truro, Cornwall, TR1 3HD, UK

关键词: Reporting;    Systematic review;    Individual participant (patient) data;    Prognosis;    Prognostic factor;    Meta-analysis;   

Others  :  1136699
DOI  :  10.1186/1471-2288-12-56

 received in 2011-08-09, accepted in 2012-04-24,  发布年份 2012

【 摘 要 】

Background

Prognostic factors are associated with the risk of a subsequent outcome in people with a given disease or health condition. Meta-analysis using individual participant data (IPD), where the raw data are synthesised from multiple studies, has been championed as the gold-standard for synthesising prognostic factor studies. We assessed the feasibility and conduct of this approach.

Methods

A systematic review to identify published IPD meta-analyses of prognostic factors studies, followed by detailed assessment of a random sample of 20 articles published from 2006. Six of these 20 articles were from the IMPACT (International Mission for Prognosis and Analysis of Clinical Trials in traumatic brain injury) collaboration, for which additional information was also used from simultaneously published companion papers.

Results

Forty-eight published IPD meta-analyses of prognostic factors were identified up to March 2009. Only three were published before 2000 but thereafter a median of four articles exist per year, with traumatic brain injury the most active research field. Availability of IPD offered many advantages, such as checking modelling assumptions; analysing variables on their continuous scale with the possibility of assessing for non-linear relationships; and obtaining results adjusted for other variables. However, researchers also faced many challenges, such as large cost and time required to obtain and clean IPD; unavailable IPD for some studies; different sets of prognostic factors in each study; and variability in study methods of measurement. The IMPACT initiative is a leading example, and had generally strong design, methodological and statistical standards. Elsewhere, standards are not always as high and improvements in the conduct of IPD meta-analyses of prognostic factor studies are often needed; in particular, continuous variables are often categorised without reason; publication bias and availability bias are rarely examined; and important methodological details and summary results are often inadequately reported.

Conclusions

IPD meta-analyses of prognostic factors are achievable and offer many advantages, as displayed most expertly by the IMPACT initiative. However such projects face numerous logistical and methodological obstacles, and their conduct and reporting can often be substantially improved.

【 授权许可】

   
2012 Abo-Zaid et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

【 参考文献 】

• [1]Steyerberg EW, Mushkudiani N, Perel P, Butcher I, Lu J, McHugh GS, Murray GD, Marmarou A, Roberts I, Habbema JDF, et al.: Predicting outcome after traumatic brain injury: Development and international validation of prognostic scores based on admission characteristics. PLoS Med 2008, 5(8):1251-1261.
• [2]Look MP, van Putten WL, Duffy MJ, Harbeck N, Christensen IJ, Thomssen C, Kates R, Spyratos F, Ferno M, Eppenberger-Castori S, et al.: Pooled analysis of prognostic impact of urokinase-type plasminogen activator and its inhibitor PAI-1 in 8377 breast cancer patients. J Natl Cancer Inst 2002, 94(2):116-128.
• [3]Sauerbrei W: Covariates. Wiley Encyclopedia of Clinical Trials 2007, 1-13.
• [4]Maas AI, Marmarou A, Murray GD, Teasdale SG, Steyerberg EW: Prognosis and clinical trial design in traumatic brain injury: the IMPACT study. J Neurotrauma 2007, 24(2):232-238.
• [5]Royston P, Altman DG, Sauerbrei W: Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 2006, 25(1):127-141.
• [6]Royston P, Moons KGM, Altman DG, Vergouwe Y: Prognosis and prognostic research: developing a prognostic model. Br Med J 2009, 338(b604):1373-1377.
• [7]Cianfrocca M, Goldstein LJ: Prognostic and predictive factors in early-stage breast cancer. Oncologist 2004, 9(6):606-616.
• [8]Riley RD, Abrams KR, Sutton AJ, Lambert PC, Jones DR, Heney D, Burchill SA: Reporting of prognostic markers: current problems and development of guidelines for evidence-based practice in the future. Br J Cancer 2003, 88(8):1191-1198.
• [9]Holländer N, Sauerbrei W: On statistical approaches for the multivariable analysis of prognostic factor studies. In Advances in statistical methods for the health sciences. Edited by Auget J-L, Balakrishnan N, Mesbah M, Molenberghs G. Birkhäuser, Boston; 2006:19-38.
• [10]Kyzas PA, Loizou KT, Ioannidis JP: Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst 2005, 97(14):1043-1055.
• [11]Sutcliffe P, Hummel S, Simpson E, Young T, Rees A, Wilkinson A, Hamdy F, Clarke N, Staffurth J: Use of classical and novel biomarkers as prognostic risk factors for localised prostate cancer: a systematic review. Health Technol Assess 2009, 13(5):1-219. iii, xi-xiii
• [12]Stewart LA, Parmar MK: Meta-analysis of the literature or of individual patient data: is there a difference? Lancet 1993, 341(8842):418-422.
• [13]Riley RD, Sauerbrei W, Altman DG: Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond. Br J Cancer 2009, 100(8):1219-1229.
• [14]Trivella M, Pezzella F, Pastorino U, Harris AL, Altman DG: Microvessel density as a prognostic factor in non-small-cell lung carcinoma: a meta-analysis of individual patient data. Lancet Oncol 2007, 8(6):488-499.
• [15]Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C: Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 1999, 28(1):1-9.
• [16]Riley RD, Lambert PC, Abo-Zaid G: Meta-analysis of individual participant data: conduct, rationale and reporting. BMJ 2010, 340:c221.
• [17]Altman DG, Trivella M, Pezzella F, Harris AL, Pastorino U: Systematic review of multiple studies of prognosis: the feasibility of obtaining individual patient data. I. In Advances in statistical methods for the health sciences. Edited by Auget J-L, Balakrishnan N, Mesbah M, Molenberghs G. Birkhäuser, Boston; 2006:3-18.
• [18]Meert AP, Paesmans M, Martin B, Delmotte P, Berghmans T, Verdebout JM, Lafitte JJ, Mascaux C, Sculier JP: The role of microvessel density on the survival of patients with lung cancer: a systematic review of the literature with meta-analysis. Br J Cancer 2002, 87(7):694-701.
• [19]Riley RD: Commentary: Like it and lump it? Meta-analysis using individual participant data. Int J Epidemiol 2010, 39(5):1359-1361.
• [20]Ioannidis JP, Rosenberg PS, Goedert JJ, O'Brien TR: Commentary: meta-analysis of individual participants' data in genetic epidemiology. Am J Epidemiol 2002, 156(3):204-210.
• [21]Moons KG, Altman DG, Vergouwe Y, Royston P: Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009, 338:b606.
• [22]Hemingway H, Riley RD, Altman DG: Ten steps towards improving prognosis research. BMJ 2009, 339:b4184.
• [23]Riley RD, Simmonds MC, Look MP: Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. J Clin Epidemiol 2007, 60(5):431-439.
• [24]Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB: Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000, 283(15)):2008-2012.
• [25]Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999, 354(9193)):1896-1900.
• [26]Marmarou A, Lu J, Butcher I, McHugh GS, Mushkudiani NA, Murray GD, Steyerberg EW, Maas AI: IMPACT database of traumatic brain injury: design and description. J Neurotrauma 2007, 24(2):239-250.
• [27]MeRGE collaborators: Independent prognostic importance of a restrictive left ventricular filling pattern after myocardial infarction: an individual patient meta-analysis. Circulation 2008, 117(20)):2591-2598.
• [28]Murray GD, Butcher I, McHugh GS, Lu J, Mushkudiani NA, Maas AIR, Marmarou A, Steyerberg EW: Multivariable prognostic analysis in traumatic brain injury: Results from the IMPACT study. J Neurotrauma 2007, 24(2):p329-p337.
• [29]Butcher I, Maas AIR, Lu J, Marmarou A, Murray GD, Mushkudiani NA, McHugh GS, Steyerberg EW: Prognostic value of admission blood pressure in traumatic brain injury: Results from the IMPACT study. J Neurotrauma 2007, 24(2):294-302.
• [30]Van Beek JGM, Mushkudiani NA, Steyerberg EW, Butcher I, McHugh GS, Lu J, Marmarou A, Murray GD, Maas AIR: Prognostic value of admission laboratory parameters in traumatic brain injury: Results from the IMPACT study. J Neurotrauma 2007, 24(2):315-328.
• [31]Maas AIR, Steyerberg EW, Butcher I, Dammers R, Lu J, Marmarou A, Mushkudiani NA, McHugh GS, Murray GD: Prognostic value of computerized tomography scan characteristics in traumatic brain injury: Results from the IMPACT study. J Neurotrauma 2007, 24(2):303-314.
• [32]Mushkudiani NA, Engel DC, Steyerberg EW, Butcher I, Lu J, Marmarou A, Slieker F, McHugh GS, Murray GD, Maas AIR: Prognostic value of demographic characteristics in traumatic brain injury: Results from the IMPACT study. J Neurotrauma 2007, 24(2):259-269.
• [33]McHugh GS, Engel DC, Butcher I, Steyerberg EW, Lu J, Mushkudiani N, Hernandez AV, Marmarou A, Maas AIR, Murray GD: Prognostic value of secondary insults in traumatic brain injury: Results from the IMPACT study. J Neurotrauma 2007, 24(2):287-293.
• [34]Butcher I, McHugh GS, Lu J, Steyerberg EW, Hernandez AV, Mushkudiani N, Maas AI, Marmarou A, Murray GD: Prognostic value of cause of injury in traumatic brain injury: results from the IMPACT study. J Neurotrauma 2007, 24(2):281-286.
• [35]Marmarou A, Lu J, Butcher I, McHugh GS, Murray GD, Steyerberg EW, Mushkudiani NA, Choi S, Maas AI: Prognostic value of the Glasgow Coma Scale and pupil reactivity in traumatic brain injury assessed pre-hospital and on enrollment: an IMPACT analysis. J Neurotrauma 2007, 24(2):270-280.
• [36]Yap YG, Duong T, Bland M, Malik M, Torp-Pedersen C, Kober L, Connolly SJ, Gallagher MM, Camm AJ: Potential demographic and baselines variables for risk stratification of high-risk post-myocardial infarction patients in the era of implantable cardioverter-defibrillator - A prognostic indicator. Int J Cardiol 2008, 126(1):101-107.
• [37]Thakkinstian A, Dmitrienko S, Gerbase-DeLima M, McDaniel DO, Inigo P, Chow KM, McEvoy M, Ingsathit A, Trevillian P, Barber WH, et al.: Association between cytokine gene polymorphisms and outcomes in renal transplantation: A meta-analysis of individual patient data. Nephrol Dial Transplant 2008, 23(9):3017-3023.
• [38]Koopman L, Hoes AW, Glasziou PP, Appelman CL, Burke P, McCormick DP, Damoiseaux RA, Le SN, Rovers MM: Antibiotic therapy to prevent the development of asymptomatic middle ear effusion in children with acute otitis media: A meta-analysis of individual patient data. Arch Otolaryngol Head Neck Surg 2008, 134(2):128-132.
• [39]Goetz CG, Wuu J, McDermott MP, Adler CH, Fahn S, Freed CR, Hauser RA, Olanow WC, Shoulson I, Tandon PK, et al.: Placebo response in Parkinson's disease: Comparisons among 11 trials covering medical and surgical interventions. Mov Disord 2008, 23(5):690-699.
• [40]MeRGE collaborators: Independence of restrictive filling pattern and LV ejection fraction with mortality in heart failure: An individual patient meta-analysis. Eur J Heart Fail 2008, 10(8)):786-792.
• [41]Sylaja PN, Dong W, Grotta JC, Miller MK, Tomita K, Hamilton S, Semba C, Hill MD: Safety outcomes of Alteplase among acute ischemic stroke patients with special characteristics. Neurocrit Care 2007, 6(3):181-185.
• [42]Schaich M, Schlenk RF, Al-Ali HK, Dohner H, Ganser A, Heil G, Illmer T, Krahl R, Krauter J, Sauerland C, et al.: Prognosis of acute myeloid leukemia patients up to 60 years of age exhibiting trisomy 8 within a non-complex karyotype: Individual patient data-based meta-analysis of the German Acute Myeloid Leukemia Intergroup. Haematologica 2007, 92(6):763-770.
• [43]Rovers MM, Glasziou P, Appelman CL, Burke P, McCormick DP, Damoiseaux RA, Little P, Le SN, Hoes AW: Predictors of pain and/or fever at 3 to 7 days for children with acute otitis media not treated initially with antibiotics: A meta-analysis of individual patient data. Pediatrics 2007, 119(3):579-585.
• [44]Noordzij JP, Lee SL, Bernet VJ, Payne RJ, Cohen SM, McLeod IK, Hier MP, Black MJ, Kerr PD, Richards ML, et al.: Early Prediction of Hypocalcemia after Thyroidectomy using Parathyroid Hormone: An Analysis of Pooled Individual Patient Data from Nine Observational Studies. J Am Coll Surg 2007, 205(6):748-754.
• [45]Lanterna LA, Ruigrok Y, Alexander S, Tang J, Biroli F, Dunn LT, Poon WS: Meta-analysis of APOE genotype and subarachnoid hemorrhage: Clinical outcome and delayed ischemia. Neurology 2007, 69(8):766-775.
• [46]Downing M, Lau F, Lesperance M, Karlson N, Shaw J, Kuziemsky C, Bernard S, Hanson L, Olajide L, Head B, et al.: Meta-analysis of survival prediction with Palliative Performance Scale. J Palliat Care 2007, 23(4):245-252.
• [47]Warkentin TE, Sheppard J-A, Sigouin CS, Kohlmann T, Eichler P, Greinacher A: Gender imbalance and risk factor interactions in heparin-induced thrombocytopenia. Blood 2006, 108(9):2937-2941.
• [48]McHugh GS, Butcher I, Steyerberg EW, Lu J, Mushkudiani N, Marmarou A, Maas AI, Murray GD: Statistical approaches to the univariate prognostic analysis of the IMPACT database on traumatic brain injury. J Neurotrauma 2007, 24(2):251-258.
• [49]Hayden JA, Cote P, Bombardier C: Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med 2006, 144(6):427-437.
• [50]Higgins JPT, Green S (Eds): Cochrane Handbook for Systematic Reviews of Interventions. John Wiley & Sons, Chichester; 2008.
• [51]McKibbon KA: Evidence-based practice. Bull Med Libr Assoc 1998, 86(3):396-401.
• [52]Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F: Methods for Meta-analysis in Medical Research. John Wiley, London; 2000.
• [53]Thompson SG, Kaptoge S, White I, Wood A, Perry P, Danesh J: Statistical methods for the time-to-event analysis of individual participant data from multiple epidemiological studies. Int J Epidemiol 2010, 39(5):1345-1359.
• [54]Higgins JP, Thompson SG, Deeks JJ, Altman DG: Measuring inconsistency in meta-analyses. BMJ 2003, 327(7414):557-560.
• [55]DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin Trials 1986, 7(3):177-188.
• [56]Egger M, Davey Smith G, Schneider M, Minder C: Bias in meta-analysis detected by a simple, graphical test. BMJ 1997, 315(7109):629-634.
• [57]Riley RD, Steyerberg EW: Meta-analysis of a binary outcome using individual participant data and aggregate data. Journal of Research Synthesis Methods 2010, 1:2-9.
• [58]Broeze KA, Opmeer BC, Bachmann LM, Broekmans FJ, Bossuyt PM, Coppus SF, Johnson NP, Khan KS, ter Riet G, van der Veen F, et al.: Individual patient data meta-analysis of diagnostic and prognostic studies in obstetrics, gynaecology and reproductive medicine. BMC Med Res Methodol 2009, 9:22. BioMed Central Full Text
• [59]Sauerbrei W: Prognostic Factors - Confusion caused by bad quality of design, analysis and reporting of many studies. In Current Research in Head and Neck CancerAdvances in Oto-Rhino-Laryngology. Volume 62. Edited by Bier H. Karger, Basel; 2005:184-200.
• [60]Sauerbrei W, Blettner M, Royston P: On alcohol consumption and all-cause mortality. J Clin Epidemiol 2001, 54:537-538.
• [61]Sauerbrei W, Holländer N, Riley RD, Altman DG: Evidence-based assessment and application of prognostic markers: the long way from single studies to meta-analysis. Communications in Statistics 2006, 35:1333-1342.
• [62]Moher D, Liberati A, Tetzlaff J, Altman DG, for the PRISMA Group): Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009, 339:b2535.
• [63]McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM, Diagnostics ftSSotN-EWoC: REporting recommendations for tumor MARKer prognostic studies (REMARK). J Natl Cancer Inst 2005, 97(16)):1180-1184.
• [64]Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG: A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med 2000, 19(24):3417-3432.
• [65]Higgins JP, Whitehead A, Turner RM, Omar RZ, Thompson SG: Meta-analysis of continuous outcome data from individual patients. Stat Med 2001, 20(15):2219-2241.
• [66]Tudur-Smith C, Williamson PR, Marson AG: Investigating heterogeneity in an individual patient data meta-analysis of time to event outcomes. Stat Med 2005, 24(9):1307-1319.
• [67]Steyerberg EW: Clinical prediction models: a practical approach to development, validation, and updating. Springer, New York; 2009.
• [68]Riley RD, Higgins JP, Deeks JJ: Interpretation of random effects meta-analyses. BMJ 2011, 342:d549.
• [69]Higgins JP, Thompson SG, Spiegelhalter DJ: A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society, Series A 2009, 172:137-159.
• [70]Ahmed I, Sutton AJ, Riley RD: Assessment of publication bias, selection bias and unavailable data in meta-analyses using individual participant data: a database survey. BMJ 2012, 344:d7762.
• [71]Kyzas PA, Denaxa-Kyza D, Ioannidis JP: Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer 2007, 43(17):2559-2579.
• [72]Sterne JAC, Terrin N, Jones DR, Lau J, Carpenter J, Rücker G, Harbord RM, Schmid CH, A.J. S J.P.A. I, et al.: Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 2011, 342:d4002.
• [73]McShane LM, Altman DG, Sauerbrei W: Identification of clinically useful cancer prognostic factors: what are we missing? J Natl Cancer Inst 2005, 97(14):1023-1025.