Dangerous predictions: evaluation methods for and consequences of predicting dangerous behavior

violence and dangerous behavior;prediction;base rates;receiver operating characteristic (ROC) and area under the ROC curve (AUC) analysis;positive and negative values;sensitivity and specificity;variance-bias trade-off

Bokhari, Ehsan

关键词: violence and dangerous behavior;    prediction;    base rates;    receiver operating characteristic (ROC) and area under the ROC curve (AUC) analysis;    positive and negative values;    sensitivity and specificity;    variance-bias trade-off;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/50561/Ehsan_Bokhari.pdf?sequence=1&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

This thesis focuses on prediction in the social sciences. We begin by discussing the “clinical efficiency” of prediction methods as defined by Meehl and Rosen (1955), and present three equivalent conditions for assessing clinical efficiency: (1) a probabilistic inequality from Meehl and Rosen; (2) an equivalent inequality given by Dawes (1962); and (3) a more flexible and easily computed inequality that we refer to as the Bokhari-Hubert condition. The misuse of the area under the receiver operating characteristic (ROC) curve (AUC) is discussed, particularly when base rates are low. The biases associated with the AUC are examined with a recommendation that the positive and negative predictive values deserve more emphasis than typically provided in the literature. A thorough review is given for cross-validation, an important but often ignored evaluation strategy in developing predictive models. The bias-variance trade-off in prediction is explained, and several shrinkage estimators are examined. To facilitate our discussion, illustrative examples using predictive methods in criminology are provided and several are extensively examined. A detailed history of predicting dangerous and violent behavior is also given. As a final conclusion, great caution should be exercised when predicting outcomes with serious social justice consequences.