Toward a Quantitative Trustworthy Evaluation of Post-Hoc XAI Feature Importance Maps Using Saliency-Based Occlusion

Abstract

The increasing interest in eXplainable Artificial Intelligence (XAI) is driven by the need to understand complex deep learning models, especially in critical fields like skin cancer classification. Existing research has provided various methods to interpret and clarify the decision-making processes of complex AI systems. Among the most widely used XAI methods for deep learning models are post-hoc saliency maps, which highlight the features that contribute most to a particular prediction. However, the trustworthiness of these explanations remains a concern, particularly when interpretations can be subjective. This raises a critical question: how can we effectively evaluate the quality of explanations generated by XAI methods for deep learning predictions and how it is possible to select the appropriate saliency map method? Consequently, this issue has caused the need to develop methods for evaluating XAI. These methods aim to not only interpret model decisions but also to compare different explanation methods through qualitative and quantitative measures. This study contributes to quantitatively evaluate and compare the performances of four saliency mapping-based XAI methods, namely LIME, SHAP, Attention Maps, and Grad-CAM. In our proposal, the outputs of XAI methods are used to create occluded images using feature importance scores. The masked images will then be fed to the end-to-end classifier. To measure the performance, the main metrics that we used to assess the XAI method faithfulness are the correlation between the classifiers prediction and the features importance score before and after occlusion. The obtained results show that SHAP outperforms the other three methods and is thus more faithful. These results may help indicate that SHAP is the most suitable XAI method that can explain skin lesion classification through InceptionResnetV2 model.