From local counterfactuals to global feature importance: efficient, robust, and model-agnostic explanations for brain connectivity networks

Background: Explainable artificial intelligence (XAI) is a technology that can enhance trust in mental state classifications by providing explanations for the reasoning behind artificial intelligence (AI) models out-puts, especially for high-dimensional and highly-correlated brain signals. Feature importance and coun-terfactual explanations are two common approaches to generate these explanations, but both have draw-backs. While feature importance methods, such as shapley additive explanations (SHAP), can be compu-tationally expensive and sensitive to feature correlation, counterfactual explanations only explain a single outcome instead of the entire model.Methods: To overcome these limitations, we propose a new procedure for computing global feature im-portance that involves aggregating local counterfactual explanations. This approach is specifically tailored to fMRI signals and is based on the hypothesis that instances close to the decision boundary and their counterfactuals mainly differ in the features identified as most important for the downstream classifica-tion task. We refer to this proposed feature importance measure as Boundary Crossing Solo Ratio (BoC-SoR), since it quantifies the frequency with which a change in each feature in isolation leads to a change in classification outcome, i.e., the crossing of the model's decision boundary.Results and Conclusions: Experimental results on synthetic data and real publicly available fMRI data from the Human Connect project show that the proposed BoCSoR measure is more robust to feature correlation and less computationally expensive than state-of-the-art methods. Additionally, it is equally effective in providing an explanation for the behavior of any AI model for brain signals. These properties are crucial for medical decision support systems, where many different features are often extracted from the same physiological measures and a gold standard is absent. Consequently, computing feature impor-tance may become computationally expensive, and there may be a high probability of mutual correlation among features, leading to unreliable results from state-of-the-art XAI methods.(c) 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )