The evaluation of the fidelity of eXplainable Artificial Intelligence (XAI) methods to their underlying models is a challenging task, primarily due to the absence of a ground truth for explanations. However, assessing fidelity is a necessary step for ensuring a correct XAI methodology. In this study, we conduct a fair and objective comparison of the current state-of-the-art XAI methods by introducing three novel image datasets with reliable ground truth for explanations. The primary objective of this comparison is to identify methods with low fidelity and eliminate them from further research, thereby promoting the development of more trustworthy and effective XAI techniques. Our results demonstrate that XAI methods based on the direct gradient calculation and the backpropagation of output information to input yield higher accuracy and reliability compared to methods relying on perturbation based or Class Activation Maps (CAM). However, these methods tend to generate more noisy saliency maps. These findings have significant implications for the advancement of XAI methods, enabling the elimination of erroneous explanations and fostering the development of more robust and reliable XAI.

中文翻译：

---

评估 XAI 事后技术的保真度：与地面真实解释数据集的比较研究


评估可解释人工智能（XAI）方法对其底层模型的保真度是一项具有挑战性的任务，主要是因为缺乏解释的基本事实。然而，评估保真度是确保 XAI 方法正确的必要步骤。在这项研究中，我们通过引入三个具有可靠地面事实解释的新颖图像数据集，对当前最先进的 XAI 方法进行公平客观的比较。此次比较的主要目的是识别保真度低的方法，并将其从进一步的研究中消除，从而促进更值得信赖和有效的 XAI 技术的开发。我们的结果表明，与依赖于基于扰动或类激活图 (CAM) 的方法相比，基于直接梯度计算和输出信息反向传播到输入的 XAI 方法可产生更高的准确性和可靠性。然而，这些方法往往会生成更多噪声的显着图。这些发现对 XAI 方法的进步具有重大意义，能够消除错误的解释并促进更强大和可靠的 XAI 的开发。