Explainability agreement between dermatologists and five visual explanations techniques in deep neural networks for melanoma AI classification

Frontiers in Medicine

Medicine

Birajara Machado¹  Aline Lissa Okita¹  William Gois Vitor¹  Raquel Machado Sousa¹  Victor Fornasiero de Paiva¹  Mara Giavina-Bianchi² 

[1] Department of Big Data, Hospital Israelita Albert Einstein, São Paulo, Brazil;null;

关键词: melanoma;    Grad-CAM;    Grad-CAM++;    Eigen-CAM;    Score-CAM;    LIME;    explainability;   

DOI  :  10.3389/fmed.2023.1241484

 received in 2023-06-16, accepted in 2023-08-14,  发布年份 2023

来源: Frontiers

【 摘 要 】

IntroductionThe use of deep convolutional neural networks for analyzing skin lesion images has shown promising results. The identification of skin cancer by faster and less expensive means can lead to an early diagnosis, saving lives and avoiding treatment costs. However, to implement this technology in a clinical context, it is important for specialists to understand why a certain model makes a prediction; it must be explainable. Explainability techniques can be used to highlight the patterns of interest for a prediction.MethodsOur goal was to test five different techniques: Grad-CAM, Grad-CAM++, Score-CAM, Eigen-CAM, and LIME, to analyze the agreement rate between features highlighted by the visual explanation maps to 3 important clinical criteria for melanoma classification: asymmetry, border irregularity, and color heterogeneity (ABC rule) in 100 melanoma images. Two dermatologists scored the visual maps and the clinical images using a semi-quantitative scale, and the results were compared. They also ranked their preferable techniques.ResultsWe found that the techniques had different agreement rates and acceptance. In the overall analysis, Grad-CAM showed the best total+partial agreement rate (93.6%), followed by LIME (89.8%), Grad-CAM++ (88.0%), Eigen-CAM (86.4%), and Score-CAM (84.6%). Dermatologists ranked their favorite options: Grad-CAM and Grad-CAM++, followed by Score-CAM, LIME, and Eigen-CAM.DiscussionSaliency maps are one of the few methods that can be used for visual explanations. The evaluation of explainability with humans is ideal to assess the understanding and applicability of these methods. Our results demonstrated that there is a significant agreement between clinical features used by dermatologists to diagnose melanomas and visual explanation techniques, especially Grad-Cam.

【 授权许可】

Unknown   
Copyright © 2023 Giavina-Bianchi, Vitor, Fornasiero de Paiva, Okita, Sousa and Machado.