Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion

Applied Sciences

Hyekyoung Hwang¹  Jitae Shin¹  Eunbyung Park¹ 

[1] Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Korea;

关键词: computer vision;    deep learning;    convolution neural network;    explainable AI;   

DOI  :  10.3390/app12031523

来源: DOAJ

【 摘 要 】

Despite increasing interest in developing interpretable machine learning methods, most recent studies have provided explanations only for single instances, require additional datasets, and are sensitive to hyperparameters. This paper proposes a confusion graph that reveals model weaknesses by constructing a confusion dictionary. Unlike other methods, which focus on the performance variation caused by single-neuron suppression, it defines the role of each neuron in two different perspectives: ‘correction’ and ‘violation’. Furthermore, our method can identify the class relationships in similar positions at the feature level, which can suggest improvements to the model. Finally, the proposed graph construction is model-agnostic and does not require additional data or tedious hyperparameter tuning. Experimental results show that the information loss from omitting the channels guided by the proposed graph can result in huge performance degradation, from 91% to 33%, while the proposed graph only retains 1% of total neurons.

【 授权许可】

Unknown