Deep convolutional neural networks (CNNs) have been widely used for fundus image classification and have achieved very impressive performance. However, the explainability of CNNs is poor because of their black-box nature, which limits their application in clinical practice. In this paper, we propose a novel method to search for discriminative regions to increase the confidence of CNNs in the classification of features in specific category, thereby helping users understand which regions in an image are important for a CNN to make a particular prediction. In the proposed method, a set of superpixels is selected in an evolutionary process, such that discriminative regions can be found automatically. Many experiments are conducted to verify the effectiveness of the proposed method. The average drop and average increase obtained with the proposed method are 0 and 77.8%, respectively, in fundus image classification, indicating that the proposed method is very effective in identifying discriminative regions. Additionally, several interesting findings are reported: 1) Some superpixels, which contain the evidence used by humans to make a certain decision in practice, can be identified as discriminative regions via the proposed method; 2) The superpixels identified as discriminative regions are distributed in different locations in an image rather than focusing on regions with a specific instance; and 3) The number of discriminative superpixels obtained via the proposed method is relatively small. In other words, a CNN model can employ a small portion of the pixels in an image to increase the confidence for a specific category.

中文翻译：

---

使用遗传算法在眼底图像分类中搜索卷积神经网络的判别区域


深度卷积神经网络 （CNN） 已广泛用于眼底图像分类，并取得了非常令人印象深刻的性能。然而，由于 CNN 的黑盒性质，其可解释性很差，这限制了它们在临床实践中的应用。在本文中，我们提出了一种搜索判别区域的新方法，以增加 CNN 对特定类别特征进行分类的置信度，从而帮助用户了解图像中的哪些区域对 CNN 做出特定预测很重要。在所提出的方法中，在进化过程中选择了一组超像素，这样就可以自动找到判别区域。进行了大量实验以验证所提方法的有效性。在眼底图像分类中，所提方法获得的平均下降和平均增加分别为 0 和 77.8%，表明所提方法在识别判别区域方面非常有效。此外，还报告了几个有趣的发现：1） 一些超像素包含人类在实践中做出特定决定的证据，可以通过所提出的方法被识别为判别区域;2） 被识别为判别区域的超像素分布在图像中的不同位置，而不是集中在具有特定实例的区域;3） 通过所提出的方法获得的判别性超像素的数量相对较小。换句话说，CNN 模型可以使用图像中的一小部分像素来提高特定类别的置信度。