A graphical tool for locating inconsistency in network meta-analyses

BMC Medical Research Methodology

Jochem König¹  Harald Binder¹  Ulrike Krahn¹ 

[1] Division Medical Biometry, Institute of Medical Biostatistics, Epidemiology and Informatics (IMBEI), University Medical Center Johannes Gutenberg University Mainz, Obere Zahlbacher Str. 69, 55131 Mainz, Germany

关键词: Hat matrix;    Cochran’s Q;    Inconsistency;    Network meta-analysis;   

Others  :  1126041
DOI  :  10.1186/1471-2288-13-35

 received in 2012-09-28, accepted in 2013-02-25,  发布年份 2013

【 摘 要 】

Background

In network meta-analyses, several treatments can be compared by connecting evidence from clinical trials that have investigated two or more treatments. The resulting trial network allows estimating the relative effects of all pairs of treatments taking indirect evidence into account. For a valid analysis of the network, consistent information from different pathways is assumed. Consistency can be checked by contrasting effect estimates from direct comparisons with the evidence of the remaining network. Unfortunately, one deviating direct comparison may have side effects on the network estimates of others, thus producing hot spots of inconsistency.

Methods

We provide a tool, the net heat plot, to render transparent which direct comparisons drive each network estimate and to display hot spots of inconsistency: this permits singling out which of the suspicious direct comparisons are sufficient to explain the presence of inconsistency. We base our methods on fixed-effects models. For disclosure of potential drivers, the plot comprises the contribution of each direct estimate to network estimates resulting from regression diagnostics. In combination, we show heat colors corresponding to the change in agreement between direct and indirect estimate when relaxing the assumption of consistency for one direct comparison. A clustering procedure is applied to the heat matrix in order to find hot spots of inconsistency.

Results

The method is shown to work with several examples, which are constructed by perturbing the effect of single study designs, and with two published network meta-analyses. Once the possible sources of inconsistencies are identified, our method also reveals which network estimates they affect.

Conclusion

Our proposal is seen to be useful for identifying sources of inconsistencies in the network together with the interrelatedness of effect estimates. It opens the way for a further analysis based on subject matter considerations.

【 授权许可】

   
2013 Krahn et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Wells GA, Sultan SA, Chen L, Khan M, Coyle D (Eds): Indirect Evidence: Indirect Treatment Comparisons in Meta-Analysis. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2009.
• [2]Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, Boersma C, Thompson D, Larholt KM, Diaz M, Barrett A: Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR task force on indirect treatment comparisons good research practices: part 2. Value Health 2011, 14(4):429-437. [http://dx.doi.org/10.1016/j.jval.2011.01.011 webcite]
• [3]Dias S, Welton NJ, Sutton AJ, E AA (Eds): A Generalised Linear Modelling Framework for Pairwise and Network Meta-Analysis of Randomised Controlled Trials,. NICE DSU: Technical Support Document 2; 2011. [http://www.nicedsu.org.uk webcite]
• [4]Salanti G: Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Syn Meth 2012, 3(2):80-97. [http://doi.wiley.com/10.1002/jrsm.1037 webcite]
• [5]Baker SG, Kramer BS: The transitive fallacy for randomized trials: if A bests B and B bests C in separate trials, is A better than C? BMC Med Res Methodol 2002, 2:13. BioMed Central Full Text
• [6]Salanti G, Marinho V, Higgins JPT: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol 2009, 62(8):857—864. [http://dx.doi.org/10.1016/j.jclinepi.2008.10.001 webcite]
• [7]Jorgensen AW, Maric KL, Tendal B, Faurschou A, Gotzsche PC: Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: differences in methodological quality and conclusions. BMC Med Res Methodol 2008, 8:60. [http://dx.doi.org/10.1186/1471-2288-8-60 webcite] BioMed Central Full Text
• [8]Lumley T: Network meta-analysis for indirect treatment comparisons. Stat Med 2002, 21(16):2313—2324. [http://dx.doi.org/10.1002/sim.1201 webcite]
• [9]Lu G, Ades AE: Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc 2006, 101(474):447-459.
• [10]Dias S, Welton NJ, Sutton AJ, Caldwell DM, Guobing L, Ades AE (Eds): Inconsistency in Networks of Evidence Based on Randomised Controlled Trials,. NICE DSU: Technical Support Document 4; 2011. [http://www.nicedsu.org.uk webcite]
• [11]Bucher HC, Guyatt GH, Griffith LE, Walter SD: The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol 1997, 50(6):683-691.
• [12]Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JP, Churchill R, Watanabe N, Nakagawa A, Omori IM, McGuire H, Tansella M, Barbui C: Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet 2009, 373(9665):746-758. [http://dx.doi.org/10.1016/S0140-6736(09)60046 webcite-5]
• [13]Salanti G, Higgins JPT, Ades AE, Ioannidis JPA: Evaluation of networks of randomized trials. Stat Methods Med Res 2008, 17(3):279-301. [http://dx.doi.org/10.1177/0962280207080643 webcite]
• [14]Dias S, Welton NJ, Caldwell DM, Ades AE: Checking consistency in mixed treatment comparison meta-analysis. Stat Med 2010, 29(7–8):932-944. [http://dx.doi.org/10.1002/sim.3767 webcite]
• [15]Higgins JPT, Jackson D, Barrett JK, Lu G, Ades aE, White IR: Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Syn Meth 2012, 3(2):98-110. [http://doi.wiley.com/10.1002/jrsm.1044 webcite]
• [16]White IR, Barrett JK, Jackson D, Higgins JPT: Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Syn Meth 2012, 3(2):111-125. [http://doi.wiley.com/10.1002/jrsm.1045 webcite]
• [17]Senn S, Gavini F, Magrez D, Scheen A: Issues in performing a network meta-analysis. Stat Methods Med Res 2012. (Epub ahead of print). [http://dx.doi.org/10.1177/0962280211432220 webcite]
• [18]Rücker G: Network meta-analysis, electrical networks and graph theory. Res Syn Meth 2012, 3(4):312-324. [http://doi.wiley.com/10.1002/jrsm.1058 webcite]
• [19]Caldwell DM, Welton NJ, Ades AE: Mixed treatment comparison analysis provides internally coherent treatment effect estimates based on overviews of reviews and can reveal inconsistency. J Clin Epidemiol 2010, 63(8):875-882. [http://dx.doi.org/10.1016/j.jclinepi.2009.08.025 webcite]
• [20]Chatterjee S, Hadi AS: Influential Observations, High Leverage Points, and Outliers in Linear Regression. Statist Sci 1986, 1(3):379-393.
• [21]Viechtbauer W, Cheung WL: Outlier and influence diagnostics for meta-analysis. Res Syn Meth 2010, 1(2):112-125.
• [22]Gumedze FN, Jackson D: A random effects variance shift model for detecting and accommodating outliers in meta-analysis. BMC Med Res Methodol 2011, 11:19. [http://dx.doi.org/10.1186/1471-2288-11-19 webcite] BioMed Central Full Text
• [23]Lu G, Welton NJ, Higgins JPT, White IR, Ades A: Linear inference for mixed treatment comparison meta-analysis: A two-stage approach. Res Syn Meth 2011, 2:43-60.
• [24]Chung H, Lumley T: Graphical exploration of network meta-analysis data: the use of multidimensional scaling. Clin Trials 2008, 5(4):301-307. [http://dx.doi.org/10.1177/1740774508093614 webcite]
• [25]Galbraith RF: A note on graphical presentation of estimated odds ratios from several clinical trials. Stat Med 1988, 7:889-894.
• [26]Aitken AC: On least squares and linear combination of observations. Proc R Soc Edinb 1934, 55:42-48.
• [27]Gleser LJ, Olkin I: Stochastically dependent effect sizes. In The Handbook of Research Synthesis and Meta-Analysis,. Edited by Cooper H, Hedges LV, Valentine JC. New York: Russell Sage Foundation; 2009:357-376.
• [28]Jackson D, Riley R, White IR: Multivariate meta-analysis: Potential and promise. Stat Med 2011, 30(20):2481-2498. [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3470931/ webcite]
• [29]Belsley DA, Kuh E, Welsch RE: Regression Diagnostics: Identifying Influential Data and Sources of Collinearity (Wiley Series in Probability and Statistics). New Jersey: John Wiley & Sons; 2004. Ⓒ1980
• [30]Cochran W: The combination of estimates from different experiments. Biometrics 1954, 10:101-129.
• [31]Raudenbush SW, Becker BJ, Kalaian H: Modeling multivariate effect sizes. Psych Bull 1988, 103:111-120.
• [32]Borenstein M, Hedges LV, Higgins JPT, Rothstein HR: Introduction to Meta-Analysis. Chichester: John Wiley & Sons; 2009.
• [33]Song F, Clark A, Bachmann MO, Maas J: Simulation evaluation of statistical properties of methods for indirect and mixed treatment comparisons. BMC Med Res Meth 2012, 12:138. [http://www.ncbi.nlm.nih.gov/pubmed/22970794 webcite] BioMed Central Full Text
• [34]Gordon AD: Classification. London: Chapman and Hall/ CRC; 1999.
• [35]Oelke D, Janetzko H, Simon S, Neuhaus K, Keim D: Visual boosting in pixel-based visualizations. Comput Graphics Forum 30 2011, 3:871—880.
• [36]R Core Team: R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing; 2012. [http://www.R-project.org/ webcite]. [ISBN 3-900051-07-0]
• [37]Gasparrini A, Armstrong B, Kenward MG: Multivariate meta-analysis for non-linear and other multi-parameter associations. Stat Med 2012, 31(29):3821-3839.
• [38]Cooper NJ, Sutton AJ, Morris D, Ades AE, Welton NJ: Addressing between-study heterogeneity and inconsistency in mixed treatment comparisons: Application to stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Stat Med 2009, 28(14):1861-1881.
• [39]Song F, Harvey I, Lilford R: Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol 2008, 61(5):455-463. [http://www.ncbi.nlm.nih.gov/pubmed/18394538 webcite]
• [40]Thompson SG: Why sources of heterogeneity in meta-analysis should be investigated. BMJ 1994, 309(6965):1351-1355.
• [41]Salanti G, Dias S, Welton NJ, Ades AE, Golfinopoulos V, Kyrgiou M, Mauri D, Ioannidis JPA: Evaluating novel agent effects in multiple-treatments meta-regression. Stat Med 2010, 29(23):2369-2383. [http://dx.doi.org/10.1002/sim.4001 webcite]