Reducing decision errors in the paired comparison of the diagnostic accuracy of screening tests with Gaussian outcomes

BMC Medical Research Methodology

Deborah H Glueck⁴  Keith E Muller¹  Aarti Munjal⁴  Sarah M Kreidler⁴  John T Brinton⁴  Todd A Alonzo²  Brandy M Ringham³ 

[1]Department of Health Outcomes and Policy, 1329 SW 16th St., Gainesville FL 32608, USA

[2]Department of Preventive Medicine, University of Southern California, 440 E. Huntington Dr, 4th floor, Arcadia CA 91006, USA

[3]Center for Cancer Prevention and Control Research, University of California, Los Angeles, 650 Charles Young Drive South, Room A2-125 CHS, Los Angeles CA 90095, USA

[4]Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, 13001 E. 17th Place, Aurora CO 80045, USA

关键词: Receiver operating characteristic analysis;    Paired screening trial;    Power;    Type I error;    Area under the curve;    Differential verification bias;    Cancer screening;   

Others  :  866390
DOI  :  10.1186/1471-2288-14-37

 received in 2013-12-13, accepted in 2014-02-26,  发布年份 2014

【 摘 要 】

Background

Scientists often use a paired comparison of the areas under the receiver operating characteristic curves to decide which continuous cancer screening test has the best diagnostic accuracy. In the paired design, all participants are screened with both tests. Participants with suspicious results or signs and symptoms of disease receive the reference standard test. The remaining participants are classified as non-cases, even though some may have occult disease. The standard analysis includes all study participants, which can create bias in the estimates of diagnostic accuracy since not all participants receive disease status verification. We propose a weighted maximum likelihood bias correction method to reduce decision errors.

Methods

Using Monte Carlo simulations, we assessed the method’s ability to reduce decision errors across a range of disease prevalences, correlations between screening test scores, rates of interval cases and proportions of participants who received the reference standard test.

Results

The performance of the method depends on characteristics of the screening tests and the disease and on the percentage of participants who receive the reference standard test. In studies with a large amount of bias in the difference in the full areas under the curves, the bias correction method reduces the Type I error rate and improves power for the correct decision. We demonstrate the method with an application to a hypothetical oral cancer screening study.

Conclusion

The bias correction method reduces decision errors for some paired screening trials. In order to determine if bias correction is needed for a specific screening trial, we recommend the investigator conduct a simulation study using our software.

【 授权许可】

   
2014 Ringham et al.; licensee BioMed Central Ltd.

【 图 表 】

【 参考文献 】

• [1]Lingen MW: Efficacy of oral cancer screening adjunctive techniques. National Institute of Dental and Craniofacial Research, National Institutes of Health, US Department of Health and Human Services. NIH Project Number 1RC2DE020779-01. 2009
• [2]Lewin JM, D’Orsi CJ, Hendrick RE, Moss LJ, Isaacs PK, Karellas A, Cutter GR: Clinical comparison of full-field digital mammography and screen-film mammography for detection of breast cancer. Am J Roentgenol 2002, 179:671-677.
• [3]Pisano ED, Gatsonis C, Hendrick E, Yaffe M, Baum JK, Acharyya S, Conant EF, Fajardo LL, Bassett L, D’Orsi C, Jong R, Rebner M: Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med 2005, 253:1773-1783.
• [4]Glueck DH, Lamb MM, O’Donnell CI, Ringham BM, Brinton JT, Muller KE, Lewin JM: Bias in trials comparing paired continuous tests can cause researchers to choose the wrong screening modality. BMC Med Res Methodol 2009, 9:4. BioMed Central Full Text
• [5]Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, Bossuyt PM: Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999, 282:1061-1066.
• [6]Reitsma JB, Rutjes AWS, Khan KS, Coomarasamy A, Bossuyt PM: A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol 2009, 62:797-806.
• [7]Rutjes AWS, Reitsma JB, Di Nisio M, Smidt N, van Rijn J C, Bossuyt PMM: Evidence of bias and variation in diagnostic accuracy studies. Can Med Assoc J 2006, 174:469-476.
• [8]Whiting P, Rutjes AWS, Reitsma JB, Glas AS, Bossuyt PMM, Kleijnen J: Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med 2004, 140:189-202.
• [9]Ringham BM, Alonzo TA, Grunwald GK, Glueck DH: Estimates of sensitivity and specificity can be biased when reporting the results of the second test in a screening trial conducted in series. BMC Med Res Methodol 2010, 10:3. BioMed Central Full Text
• [10]Kish L: Survey Sampling . Hoboken: John Wiley & Sons; 1965.
• [11]Nath GB: Estimation in truncated bivariate normal distributions. J Roy Stat Soc C-App 1971, 20:313-319.
• [12]Ross S: A First Course in Probability . Prentice Hall: Upper Saddle River; 2009.
• [13]GitHub Repository: Bias Correction Suite. [http://www.github.com/SampleSizeShop/BiasCorrectionSuite webcite]
• [14]Metz CE, Herman BA, Shen JH: Maximum likelihood estimation of receiver operating characteristic (roc) curves from continuously-distributed data. Stat Med 1998, 17:1033-1053.
• [15]Obuchowski NA, McClish DK: Sample size determination for diagnostic accuracy studies involving binormal roc curve indices. Stat Med 1997, 16:1529-1542.
• [16]Bunker CH, Patrick AL, Konety BR, Dhir R, Brufsky AM, Vivas CA, Becich MJ, Trump DL, Kuller LH: High prevalence of screening-detected prostate cancer among afro-caribbeans: the tobago prostate cancer survey. Cancer Epidem Biomar 2002, 11:726-729.
• [17]Lim K, Moles DR, Downer MC, Speight PM: Opportunistic screening for oral cancer and precancer in general dental practice: results of a demonstration study. Brit Dent J 2003, 194:497-502.
• [18]Howlader N, Noone AM, Krapcho M, Neyman N, Aminou R, Waldron W, Altekruse SF, Kosary CL, Ruhl J, Tatalovich Z, Cho H, Mariotto A, Eisner MP, Lewis DR, Chen HS, Feuer EJ, Cronin KA, Edwards BK (Eds): SEER cancer statistics review, 1975-2008. 2011. [http://seer.cancer.gov/csr/1975_2009_pops09/ webcite]
• [19]Bobo JK, Lee NC, Thames SF: Findings from 752,081 clinical breast examinations reported to a national screening program from 1995 through 1998. J Natl Cancer I 2000, 92:971-976.
• [20]Alonzo TA: Verification bias-corrected estimators of the relative true and false positive rates of two binary screening tests. Stat Med 2005, 24:403-417.
• [21]Bureau USC: State and County Quickfacts. [http://quickfacts.census.gov webcite]
• [22]Begg CB, Greenes RA: Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983, 39:207-215.
• [23]Alonzo TA, Pepe MS: Assessing accuracy of a continuous screening test in the presence of verification bias. J Roy Stat Soc C-App 2005, 54:173-190.
• [24]Buzoianu M, Kadane JB: Adjusting for verification bias in diagnostic test evaluation: A bayesian approach. Stat Med 2008, 27:2453-2473.
• [25]Martinez EZ, Alberto Achcar J, Louzada-Neto F: Estimators of sensitivity and specificity in the presence of verification bias: a bayesian approach. Comput Stat Data An 2006, 51:601-611.
• [26]Rotnitzky A, Faraggi D, Schisterman E: Doubly robust estimation of the area under the receiver-operating characteristic curve in the presence of verification bias. J Am Stat Assoc 2006, 101:1276-1288.
• [27]Toledano AY, Gatsonis C: Generalized estimating equations for ordinal categorical data: arbitrary patterns of missing responses and missingness in a key covariate. Biometrics 1999, 55:488-496.
• [28]Zhou X: Maximum likelihood estimators of sensitivity and specificity corrected for verification bias. Commun Stat A-Theor 1993, 22:3177-3198.
• [29]Vacek PM: The effect of conditional dependence on the evaluation of diagnostic tests. Biometrics 1985, 41:959-968.
• [30]Torrance-Rynard VL, Walter SD: Effects of dependent errors in the assessment of diagnostic test performance. Stat Med 1997, 16:2157-2175.
• [31]Metz C, Kronman HA, Wang P: New approach for testing the significance of differences between roc curves measured from correlated data. In Information Processing In Medical Imaging . Edited by Deconinck F. The Hague: Springer; 1984:432-445.
• [32]Elashoff D, Zhou H, Reiss J, Wang J, Xiao H, Henson B, Hu S, Arellano M, Sinha U, Le A, Messadi D, Wang M, Nabili V, Lingen M, Morris D, Randolph T, Feng Z, Akin D, Kastratovic DA, Chia D, Abemayor E, Wong DTW: Prevalidation of salivary biomarkers for oral cancer detection. Cancer Epidem Biomar 2012, 21:664-672.