Novel methods for elucidating modality importance in multimodal electrophysiology classifiers

Frontiers in Neuroinformatics

Neuroscience

Rongen Zhang¹  Darwin A. Carbajal²  May D. Wang³  Charles A. Ellis⁴  Vince D. Calhoun⁵  Robyn L. Miller⁶  Mohammad S. E. Sendi⁷ 

[1] Hankamer School of Business, Baylor University, Waco, TX, United States;The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA, United States;The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Emory University, Atlanta, GA, United States;The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Emory University, Atlanta, GA, United States;Tri-Institutional Center for Translational Research in Neuroimaging and Data Science, Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, United States;The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Emory University, Atlanta, GA, United States;Tri-Institutional Center for Translational Research in Neuroimaging and Data Science, Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, United States;Department of Computer Science, Georgia State University, Atlanta, GA, United States;Tri-Institutional Center for Translational Research in Neuroimaging and Data Science, Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, United States;Department of Computer Science, Georgia State University, Atlanta, GA, United States;Tri-Institutional Center for Translational Research in Neuroimaging and Data Science, Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, United States;McLean Hospital and Harvard Medical School, Boston, MA, United States;

关键词: multimodal classification;    explainable deep learning;    sleep stage classification;    electrophysiology;    electroencephalography;    electrooculography;    electromyography;   

DOI  :  10.3389/fninf.2023.1123376

 received in 2022-12-14, accepted in 2023-03-01,  发布年份 2023

来源: Frontiers

【 摘 要 】

IntroductionMultimodal classification is increasingly common in electrophysiology studies. Many studies use deep learning classifiers with raw time-series data, which makes explainability difficult, and has resulted in relatively few studies applying explainability methods. This is concerning because explainability is vital to the development and implementation of clinical classifiers. As such, new multimodal explainability methods are needed.MethodsIn this study, we train a convolutional neural network for automated sleep stage classification with electroencephalogram (EEG), electrooculogram, and electromyogram data. We then present a global explainability approach that is uniquely adapted for electrophysiology analysis and compare it to an existing approach. We present the first two local multimodal explainability approaches. We look for subject-level differences in the local explanations that are obscured by global methods and look for relationships between the explanations and clinical and demographic variables in a novel analysis.ResultsWe find a high level of agreement between methods. We find that EEG is globally the most important modality for most sleep stages and that subject-level differences in importance arise in local explanations that are not captured in global explanations. We further show that sex, followed by medication and age, had significant effects upon the patterns learned by the classifier.DiscussionOur novel methods enhance explainability for the growing field of multimodal electrophysiology classification, provide avenues for the advancement of personalized medicine, yield unique insights into the effects of demographic and clinical variables upon classifiers, and help pave the way for the implementation of multimodal electrophysiology clinical classifiers.

【 授权许可】

Unknown   
Copyright © 2023 Ellis, Sendi, Zhang, Carbajal, Wang, Miller and Calhoun.