This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree (fully transparent, interpretable, white-box model), a random forest (semi-interpretable, grey-box model), and a neural network (fully opaque, black-box model). We tested the counterfactual generation process using four algorithms (DiCE, WatcherCF, prototype, and GrowingSpheresCF) in the literature in 25 different datasets. Our findings indicate that: (1) Different machine learning models have little impact on the generation of counterfactual explanations; (2) Counterfactual algorithms based uniquely on proximity loss functions are not actionable and will not provide meaningful explanations; (3) One cannot have meaningful evaluation results without guaranteeing plausibility in the counterfactual generation. Algorithms that do not consider plausibility in their internal mechanisms will lead to biased and unreliable conclusions if evaluated with the current state-of-the-art metrics; (4) A counterfactual inspection analysis is strongly recommended to ensure a robust examination of counterfactual explanations and the potential identification of biases.

本研究通过对三种不同类型的模型进行基准评估，调查机器学习模型对反事实解释生成的影响：决策树（完全透明、可解释、白盒模型）、随机森林（半可解释、灰盒模型）和神经网络（完全不透明的黑盒模型）。我们使用文献中的四种算法（DiCE、WatcherCF、prototype 和 GrowingSpheresCF）在 25 个不同的数据集中测试了反事实生成过程。我们的研究结果表明：（1）不同的机器学习模型对反事实解释的生成影响不大； (2) 唯一基于邻近损失函数的反事实算法不具有可操作性，也不会提供有意义的解释； (3) 如果不保证反事实生成的合理性，就不可能获得有意义的评估结果。如果使用当前最先进的指标进行评估，不考虑其内部机制合理性的算法将导致有偏见且不可靠的结论； (4) 强烈建议进行反事实检查分析，以确保对反事实解释进行强有力的检查并识别潜在的偏见。