Improving the error rates of the Begg and Mazumdar test for publication bias in fixed effects meta-analysis

BMC Medical Research Methodology

Ivar Heuch¹  Miriam Gjerdevik² 

[1] Department of Mathematics, University of Bergen, P. O. Box 7800, N-5020 Bergen, Norway;Department of Global Public Health and Primary Care, University of Bergen, P. O. Box 7804, N-5018 Bergen, Norway

关键词: Error rates;    Spearman’s rho;    Kendall’s tau;    Rank correlation;    Publication bias;    Meta-analysis;   

Others  :  1090914
DOI  :  10.1186/1471-2288-14-109

 received in 2014-04-29, accepted in 2014-09-17,  发布年份 2014

【 摘 要 】

Background

The rank correlation test introduced by Begg and Mazumdar is extensively used in meta-analysis to test for publication bias in clinical and epidemiological studies. It is based on correlating the standardized treatment effect with the variance of the treatment effect using Kendall’s tau as the measure of association. To our knowledge, the operational characteristics regarding the significance level of the test have not, however, been fully assessed.

Methods

We propose an alternative rank correlation test to improve the error rates of the original Begg and Mazumdar test. This test is based on the simulated distribution of the estimated measure of association, conditional on sampling variances. Furthermore, Spearman’s rho is suggested as an alternative rank correlation coefficient. The attained level and power of the tests are studied by simulations of meta-analyses assuming the fixed effects model.

Results

The significance levels of the original Begg and Mazumdar test often deviate considerably from the nominal level, the null hypothesis being rejected too infrequently. It is proven mathematically that the assumptions for using the rank correlation test are not strictly satisfied. The pairs of variables fail to be independent, and there is a correlation between the standardized effect sizes and sampling variances under the null hypothesis of no publication bias. In the meta-analysis setting, the adverse consequences of a false negative test are more profound than the disadvantages of a false positive test. Our alternative test improves the error rates in fixed effects meta-analysis. Its significance level equals the nominal value, and the Type II error rate is reduced. In small data sets Spearman’s rho should be preferred to Kendall’s tau as the measure of association.

Conclusions

As the attained significance levels of the test introduced by Begg and Mazumdar often deviate greatly from the nominal level, modified rank correlation tests, improving the error rates, should be preferred when testing for publication bias assuming fixed effects meta-analysis.

【 授权许可】

   
2014 Gjerdevik and Heuch; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

【 参考文献 】

• [1]Dickersin K, Scherer R, Lefebvre C: Identifying relevant studies for systematic reviews. Br Med J 1994, 309:1286-1291.
• [2]Begg CB: Publication bias. In The Handbook of Research Synthesis. Edited by Cooper H, Hedges L. New York: Russel Sage Foundation; 1994:399-409.
• [3]Begg CB, Berlin JA: Publication bias: a problem in interpreting medical data. J R Stat Soc Ser A 1988, 151:419-463.
• [4]Macaskill P, Walter SD, Irwig L: A comparison of methods to detect publication bias in meta-analysis. Stat Med 2001, 20:641-654.
• [5]Rücker G, Schwarzer G, Carpenter J: Arcsine test for publication bias in meta-analyses with binary outcomes. Stat Med 2008, 27:746-763.
• [6]Begg CB, Mazumdar M: Operating characteristics of a rank correlation test for publication bias. Biometrics 1994, 50:1088-1101.
• [7]Light RJ, Pillemer DB: Summing Up: The Science of Reviewing Research. Cambridge, MA: Harvard University Press; 1984.
• [8]Egger M, Smith GD, Schneider M, Minder C: Bias in meta-analysis detected by a simple, graphical test. Br Med J 1997, 315:629-634.
• [9]Higgins JPT, Thompson SG: Quantifying heterogeneity in a meta-analysis. Stat Med 2002, 21:1539-1558.
• [10]Thomson Reuters: Web of Science. [http://thomsonreuters.com/thomson-reuters-web-of-science/ webcite]
• [11]Irwig L, Macaskill P, Berry G, Glasziou P: Bias in meta-analysis detected by a simple, graphical test Graphical test is itself biased (letter). Br Med J 1998, 316:470.
• [12]Schwarzer G, Antes G, Schumacher M: Inflation of type I error rate in two statistical tests for the detection of publication bias in meta-analyses with binary outcome. Stat Med 2002, 21:2465-2477.
• [13]Harbord R, Egger M, Sterne JAC: A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med 2006, 25:3443-3457.
• [14]Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L: Comparison of two methods to detect publication bias in meta-analysis. J Am Med Assoc 2006, 295:676-660.
• [15]Kendall M, Gibbons JD: Rank Correlation Methods. London: Edward Arnold; 1990.
• [16]Lancaster HO: Significance tests in discrete distributions. J Am Stat Assoc 1961, 56:223-234.
• [17]Lindeman RH, Merenda PF, Gold RZ: Introduction to Bivariate and Multivariate Analysis. Glenview, IL: Scott Foresman; 1980.
• [18]Copas JB: A likelihood-based sensitivity analysis for publication bias in meta-analysis. J R Stat Soc Ser C (Appl Stat) 2013, 62:47-66.
• [19]Preston C, Ashby D, Smyth R: Adjusting for publication bias: modelling the selection process. J Eval Clin Prac 2004, 10:313-322.
• [20]R Development Core Team: R: A Language and Environment for Statistical Computing. [http://www.R-project.org webcite]
• [21]Mann HB: Nonparametric tests against trend. Econometrica 1945, 13:245-259.
• [22]Cottingham J, Hunter D: Chlamydia trachomatis and oral contraceptive use: A quantitative review. Genitourin Med 1992, 68:209-216.
• [23]Raudenbush SW: Magnitude of teacher expectancy effects on pupil IQ as a function of the credibility of expectancy induction: A synthesis of findings from 18 experiments. J Educ Psychol 1984, 76:85-97.
• [24]Raudenbush SW, Bryk AS: Empirical Bayes meta-analysis. J Educ Stat 1985, 10:75-98.
• [25]McLeod AI: Kendall: Kendall rank correlation and Mann-Kendall trend test. [http://cran.r-project.org/web/packages/Kendall/index.html webcite]
• [26]Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, Carpenter J, Rücker G, Harbord R, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S, Altman DG, Moher D, Higgins JPT: Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. Br Med J 2011, 343:d4002.
• [27]Bürkner PC, Doebler P: Testing for publication bias in diagnostic meta-analysis: a simulation study. Stat Med 2014, 33:3061-3077.
• [28]Baker J, Jackson D: Using journal impact factors to correct for the publiction bias of medical studies. Biometrics 2006, 62:785-792.
• [29]Bowden J, Jackson D, Thompson SG: Modelling multiple sources of dissemination bias in meta-analysis. Stat Med 2010, 29:945-955.
• [30]Moreno SG, Sutton AJ, Ades AE, Stanley TD, Abrams KR, Peters JL, Cooper NJ: Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Med Res Methodol 2009, 9:2. BioMed Central Full Text
• [31]Duval S, Tweedie R: A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc 2000, 95:89-98.
• [32]Duval S, Tweedie R: Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000, 56:455-463.
• [33]Copas J, Shi JQ: Meta-analysis, funnel plots and sensitivity analysis. Biostatistics 2000, 1:247-262.
• [34]Rücker G, Carpenter JR, Schwarzer G: Detecting and adjusting for small-study effects in meta-analysis. Biom J 2011, 53:351-368.
• [35]Stanley TD: Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection. Oxford Bull Econ Stat 2008, 70:103-127.