Adjustment for reporting bias in network meta-analysis of antidepressant trials

BMC Medical Research Methodology

Philippe Ravaud¹  Gilles Chatellier³  Ludovic Trinquart² 

[1] Assistance Publique-Hôpitaux de Paris, Hôpital Hôtel-Dieu, Centre d'Epidémiologie Clinique, Paris, France;INSERM CIE 4, Paris, France;Assistance Publique-Hôpitaux de Paris, Hôpital Européen Georges Pompidou, Unité de Recherche Clinique, Paris, France

关键词: Small-study effect;    Publication bias;    Network meta-analysis;   

Others  :  1126696
DOI  :  10.1186/1471-2288-12-150

 received in 2012-06-12, accepted in 2012-09-19,  发布年份 2012

【 摘 要 】

Background

Network meta-analysis (NMA), a generalization of conventional MA, allows for assessing the relative effectiveness of multiple interventions. Reporting bias is a major threat to the validity of MA and NMA. Numerous methods are available to assess the robustness of MA results to reporting bias. We aimed to extend such methods to NMA.

Methods

We introduced 2 adjustment models for Bayesian NMA. First, we extended a meta-regression model that allows the effect size to depend on its standard error. Second, we used a selection model that estimates the propensity of trial results being published and in which trials with lower propensity are weighted up in the NMA model. Both models rely on the assumption that biases are exchangeable across the network. We applied the models to 2 networks of placebo-controlled trials of 12 antidepressants, with 74 trials in the US Food and Drug Administration (FDA) database but only 51 with published results. NMA and adjustment models were used to estimate the effects of the 12 drugs relative to placebo, the 66 effect sizes for all possible pair-wise comparisons between drugs, probabilities of being the best drug and ranking of drugs. We compared the results from the 2 adjustment models applied to published data and NMAs of published data and NMAs of FDA data, considered as representing the totality of the data.

Results

Both adjustment models showed reduced estimated effects for the 12 drugs relative to the placebo as compared with NMA of published data. Pair-wise effect sizes between drugs, probabilities of being the best drug and ranking of drugs were modified. Estimated drug effects relative to the placebo from both adjustment models were corrected (i.e., similar to those from NMA of FDA data) for some drugs but not others, which resulted in differences in pair-wise effect sizes between drugs and ranking.

Conclusions

In this case study, adjustment models showed that NMA of published data was not robust to reporting bias and provided estimates closer to that of NMA of FDA data, although not optimal. The validity of such methods depends on the number of trials in the network and the assumption that conventional MAs in the network share a common mean bias mechanism.

【 授权许可】

   
2012 Trinquart et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

【 参考文献 】

• [1]Lumley T: Network meta-analysis for indirect treatment comparisons. Stat Med 2002, 21(16):2313-2324.
• [2]Lu G, Ades AE: Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004, 23(20):3105-3124.
• [3]Salanti G, Higgins JPT, Ades AE, Ioannidis JPA: Evaluation of networks of randomized trials. Stat Methods Med Res 2008, 17(3):279-301.
• [4]Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, et al.: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 2008, 3(8):e3081.
• [5]Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, Hing C, Kwok CS, Pang C, Harvey I: Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess 2010, 14(8):193.
• [6]Salanti G, Kavvoura FK, Ioannidis JP: Exploring the geometry of treatment networks. Ann Intern Med 2008, 148(7):544-553.
• [7]Li T, Puhan MA, Vedula SS, Singh S, Dickersin K: Network meta-analysis-highly attractive but more methodological research is needed. BMC Med 2011, 9:79. BioMed Central Full Text
• [8]Trinquart L, Abbé A, Ravaud P: Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials. PLoS One 2012, 7(4):e35219.
• [9]Hedges LV: Modeling publication selection effects in meta-analysis. Stat Sci 1992, 7(2):246-255.
• [10]Silliman NP: Hierarchical selection models with applications in meta-analysis. JASA 1997, 92(439):926-936.
• [11]Larose DT, Dey DK: Modeling publication bias using weighted distributions in a Bayesian framework. Comput Stat Data Anal 1998, 26:279-302.
• [12]Copas J, Shi JQ: Meta-analysis, funnel plots and sensitivity analysis. Biostatistics 2000, 1(3):247-262.
• [13]Duval S, Tweedie R: Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000, 56(2):455-463.
• [14]Sutton AJ, Song F, Gilbody SM, Abrams KR: Modelling publication bias in meta-analysis: a review. Stat Methods Med Res 2000, 9(5):421-445.
• [15]Copas JB, Shi JQ: A sensitivity analysis for publication bias in systematic reviews. Stat Methods Med Res 2001, 10(4):251-265.
• [16]Preston C, Ashby D, Smyth R: Adjusting for publication bias: modelling the selection process. J Eval Clin Pract 2004, 10(2):313-322.
• [17]Bowden J, Jackson D, Thompson SG: Modelling multiple sources of dissemination bias in meta-analysis. Stat Med 2010, 29(7–8):945-955.
• [18]Carpenter J, Rucker G, Schwarzer G: Assessing the sensitivity of meta-analysis to selection bias: a multiple imputation approach. Biometrics 2011, 67(3):1066-1072.
• [19]Rucker G, Carpenter JR, Schwarzer G: Detecting and adjusting for small-study effects in meta-analysis. Biom J 2011, 53(2):351-368.
• [20]Rufibach K: Selection models with monotone weight functions in meta analysis. Biom J 2011, 53(4):689-704.
• [21]Moreno SG, Sutton AJ, Ades AE, Stanley TD, Abrams KR, Peters JL, Cooper NJ: Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Med Res Methodol 2009, 9:2. BioMed Central Full Text
• [22]Moreno SG, Sutton AJ, Thompson JR, Ades AE: Abrams KR. Cooper NJ: A generalized weighting regression-derived meta-analysis estimator robust to small-study effects and heterogeneity. Stat Med; 2012.
• [23]Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R: Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008, 358(3):252-260.
• [24]Higgins JP, Whitehead A: Borrowing strength from external trials in a meta-analysis. Stat Med 1996, 15(24):2733-2749.
• [25]Lu G, Ades AE: Assessing evidence inconsistency in mixed treatment comparisons. JASA 2006, 101(474):447-459.
• [26]Lu G, Ades A: Modeling between-trial variance structure in mixed treatment comparisons. Biostatistics 2009, 10(4):792-805.
• [27]Stanley TD: Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection*. Oxf Bull Econ Stat 2008, 70(1):103-127.
• [28]Moreno SG, Sutton AJ, Turner EH, Abrams KR, Cooper NJ, Palmer TM, Ades AE: Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications. BMJ 2009, 339:b2981.
• [29]Rucker G, Schwarzer G, Carpenter JR, Binder H, Schumacher M: Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. Biostatistics 2011, 12(1):122-142.
• [30]Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ: Publication and related biases. Health Technol Assess 2000, 4(10):1-115.
• [31]Hedges LV: [Selection models and the file drawer problem]: comment. Stat Sci 1988, 3(1):118-120.
• [32]Bedrick EJ, Christensen R, Johnson W: A new perspective on priors for generalized linear models. JASA 1996, 91(436):1450-1460.
• [33]Wu Y, Shih WJ, Moore DF: Elicitation of a beta prior for Bayesian inference in clinical trials. Biom J 2008, 50(2):212-223.
• [34]Salanti G, Ades AE, Ioannidis JP: Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol 2010, 64(2):163-171.
• [35]Spiegelhalter DJ, Best NG, Carlin BP, Van Der Linde A: Bayesian measures of model complexity and fit. J R Stat Soc Ser B Stat Methodol 2002, 64(4):583-639.
• [36]Dias S, Welton NJ, Marinho VCC, Salanti G, Higgins JPT, Ades AE: Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta-analysis. J R Stat Soc Ser A Stat Soc 2010, 173(3):613-629.
• [37]Salanti G, Dias S, Welton NJ, Ades AE, Golfinopoulos V, Kyrgiou M, Mauri D, Ioannidis JP: Evaluating novel agent effects in multiple-treatments meta-regression. Stat Med 2010, 29(23):2369-2383.
• [38]Dias S, Welton NJ, Ades AE: Study designs to detect sponsorship and other biases in systematic reviews. J Clin Epidemiol 2010, 63(6):587-588.
• [39]Moreno SG, Sutton AJ, Ades AE, Cooper NJ, Abrams KR: Adjusting for publication biases across similar interventions performed well when compared with gold standard data. J Clin Epidemiol 2011, 64(11):1230-1241.
• [40]Chootrakool H, Shi JQ, Yue R: Meta-analysis and sensitivity analysis for multi-arm trials with selection bias. Stat Med 2011, 30(11):1183-1198.
• [41]Mavridis D, Sutton A, Cipriani A, Salanti G: A fully Bayesian application of the Copas selection model for publication bias extended to network meta-analysis. Stat Med 2012. [Epub ahead of print].
• [42]McCandless LC, Gustafson P, Levy AR, Richardson S: Hierarchical priors for bias parameters in Bayesian sensitivity analysis for unmeasured confounding. Stat Med 2012, 31(4):383-396.
• [43]Gelman A, Carlin JB, Stern HS, Rubin DB: Bayesian Data Analysis. 2nd edition. New York: Chapman Hall/CRC; 2004.
• [44]Rising K, Bacchetti P, Bero L: Reporting bias in drug trials submitted to the food and drug administration: review of publication and presentation. PLoS Med 2008, 5(11):e217. discussion e217.
• [45]Lathyris DN, Patsopoulos NA, Salanti G, Ioannidis JP: Industry sponsorship and selection of comparators in randomized clinical trials. Eur J Clin Invest 2010, 40(2):172-182.
• [46]Lexchin J, Bero LA, Djulbegovic B, Clark O: Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003, 326(7400):1167-1170.
• [47]Bero L, Oostvogel F, Bacchetti P, Lee K: Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others. PLoS Med 2007, 4(6):e184.
• [48]Chalmers I, Matthews R: What are the implications of optimism bias in clinical research? Lancet 2006, 367(9509):449-450.
• [49]Rucker G, Schwarzer G, Carpenter J: Arcsine test for publication bias in meta-analyses with binary outcomes. Stat Med 2008, 27(5):746-763.