Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable

BMC Medical Research Methodology

Ewout W Steyerberg²  Peter C Austin¹ 

[1] Dalla Lana School of Public Health, University of Toronto, Toronto, Canada;Department of Public Health, Erasmus Medical Centre, Rotterdam, The Netherlands

关键词: Predictive accuracy;    Predictive model;    Prediction;    Regression model;    Discrimination;    ROC curve;    Area under the receiver operating characteristic curve;    c-statistic;    Logistic regression;   

Others  :  1136605
DOI  :  10.1186/1471-2288-12-82

 received in 2011-11-23, accepted in 2012-06-11,  发布年份 2012

【 摘 要 】

Background

When outcomes are binary, the c-statistic (equivalent to the area under the Receiver Operating Characteristic curve) is a standard measure of the predictive accuracy of a logistic regression model.

Methods

An analytical expression was derived under the assumption that a continuous explanatory variable follows a normal distribution in those with and without the condition. We then conducted an extensive set of Monte Carlo simulations to examine whether the expressions derived under the assumption of binormality allowed for accurate prediction of the empirical c-statistic when the explanatory variable followed a normal distribution in the combined sample of those with and without the condition. We also examine the accuracy of the predicted c-statistic when the explanatory variable followed a gamma, log-normal or uniform distribution in combined sample of those with and without the condition.

Results

Under the assumption of binormality with equality of variances, the c-statistic follows a standard normal cumulative distribution function with dependence on the product of the standard deviation of the normal components (reflecting more heterogeneity) and the log-odds ratio (reflecting larger effects). Under the assumption of binormality with unequal variances, the c-statistic follows a standard normal cumulative distribution function with dependence on the standardized difference of the explanatory variable in those with and without the condition. In our Monte Carlo simulations, we found that these expressions allowed for reasonably accurate prediction of the empirical c-statistic when the distribution of the explanatory variable was normal, gamma, log-normal, and uniform in the entire sample of those with and without the condition.

Conclusions

The discriminative ability of a continuous explanatory variable cannot be judged by its odds ratio alone, but always needs to be considered in relation to the heterogeneity of the population.

【 授权许可】

   
2012 Austin and Steyerberg; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

【 参考文献 】

• [1]Steyerberg EW: Clinical Prediction Models. Springer, New York; 2009.
• [2]Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al.: Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010, 21:128-138.
• [3]Harrell FE Jr: Regression modeling strategies. Springer, New York, NY; 2001.
• [4]Hanley JA, McNeil BJ: The meaning and use of the area under a Receiver Operating Characteristic (ROC) curve. Radiology 1982, 143:29-36.
• [5]Bamber D: The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J Math Psychol 1975, 12:387-415.
• [6]Demler OV, Pencina MJ, D’Agostino RB Sr: Equivalence of improvement in area under ROC curve and linear discriminant analysis coefficient under assumption of normality. Statistics in Medicine 2011, 30:1410-1418.
• [7]Royston P, Altman DG: Visualizing and assessing discrimination in the logistic regression model. Statistics in Medicine 2010, 29:2508-2520.
• [8]Royston P, Thompson SG: Model-based screening by risk with application to Down’s syndrome. Statistics in Medicine 1992, 11:257-268.
• [9]Deeks JJ, Macaskill P, Irwig L: The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. Journal of Clinical Epidemiology 2005, 58:882-893.
• [10]Zhou X, Obuchowski N, McClish D: Statistical Methods in diagnostic medicine. Wiley-Interscience, New York; 2002.
• [11]Cohen J: Statistical Power Analysis for the Behavioural Sciences. 2nd edition. Lawrence Erlbaum Associates, Hillsdale, NJ; 1988.
• [12]Flury BK, Riedwyl H: Standard distance in univariate and multivariate analysis. Am Stat 1986, 40:249-251.
• [13]Austin PC: Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Statistics in Medicine 2009, 28:3083-3107.
• [14]Normand ST, Landrum MB, Guadagnoli E, Ayanian JZ, Ryan TJ, Cleary PD, et al.: Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. Journal of Clinical Epidemiology 2001, 54:387-398.
• [15]Hosmer DW, Lemeshow S: Applied Logistic Regression. John Wiley & Sons, New York, NY; 1989.
• [16]R Core Development Team: R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna; 2005.
• [17]Tu JV, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, et al.: Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. JAMA 2009, 302:2330-2337.
• [18]Tu JV, Donovan LR, Lee DS, Austin PC, Ko DT, Wang JT, et al.: Quality of Cardiac Care in Ontario. Institute for Clinical Evaluative Sciences, Toronto, Ontario; 2004.
• [19]Janssens AC, Moonesinghe R, Yang Q, Steyerberg EW, van Duijn CM, Khoury MJ: The impact of genotype frequencies on the clinical validity of genomic profiling for predicting common chronic diseases. Genetics in Medicine 2007, 9:528-535.
• [20]Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P: Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 2004, 159:882-890.
• [21]Vergouwe Y, Moons KG, Steyerberg EW: External validity of risk models: Use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Am J Epidemiol 2010, 172:971-980.