ROC Curves, Loss Functions, and Distorted Probabilities in Binary Classification

Mathematics

Phuong Bich Le¹  Zung Tien Nguyen² 

[1] Department of Mathematics, Hanoi University of Mining and Geology, No. 18 Vien Street, Duc Thang Ward, Bac Tu Liem District, Hanoi City 100000, Vietnam;Institut de Mathématiques de Toulouse, Université Paul Sabatier, 31062 Toulouse, France;

关键词: optimization;    binary classification;    machine learning;    ROC curve;    accuracy metrics;    loss function;   

DOI  :  10.3390/math10091410

来源: DOAJ

【 摘 要 】

The main purpose of this work is to study how loss functions in machine learning influence the “binary machines”, i.e., probabilistic AI models for predicting binary classification problems. In particular, we show the following results: (i) Different measures of accuracy such as area under the curve (AUC) of the ROC curve, the maximal balanced accuracy, and the maximally weighted accuracy are topologically equivalent, with natural inequalities relating them; (ii) the so-called real probability machines with respect to given information spaces are the optimal machines, i.e., they have the highest precision among all possible machines, and moreover, their ROC curves are automatically convex; (iii) the cross-entropy and the square loss are the most natural loss functions in the sense that the real probability machine is their minimizer; (iv) an arbitrary strictly convex loss function will also have as its minimizer an optimal machine, which is related to the real probability machine by just a reparametrization of sigmoid values; however, if the loss function is not convex, then its minimizer is not an optimal machine, and strange phenomena may happen.

【 授权许可】

Unknown