Deep learning algorithms can extract meaningful diagnostic features from biomedical images, promising improved patient care in digital pathology. Vision Transformer (ViT) models capture long-range spatial relationships and offer robust prediction power and better interpretability for image classification tasks than convolutional neural network models. However, limited annotated biomedical imaging datasets can cause ViT models to overfit, leading to false predictions due to random noise. To address this, we introduce Training Attention and Validation Attention Consistency (TAVAC), a metric for evaluating ViT model overfitting and quantifying interpretation reproducibility. By comparing high-attention regions between training and testing, we tested TAVAC on four public image classification datasets and two independent breast cancer histological image datasets. Overfitted models showed significantly lower TAVAC scores. TAVAC also distinguishes off-target from on-target attentions and measures interpretation generalization at a fine-grained cellular level. Beyond diagnostics, TAVAC enhances interpretative reproducibility in basic research, revealing critical spatial patterns and cellular structures of biomedical and other general nonbiomedical images.

中文翻译：

---

使用 TAVAC 量化 Vision Transformer 模型中的解释重现性


深度学习算法可以从生物医学图像中提取有意义的诊断特征，有望改善数字病理学中的患者护理。与卷积神经网络模型相比，Vision Transformer （ViT） 模型可以捕获长距离空间关系，并为图像分类任务提供强大的预测能力和更好的可解释性。然而，有限的注释生物医学成像数据集会导致 ViT 模型过度拟合，从而导致随机噪声导致错误预测。为了解决这个问题，我们引入了训练注意力和验证注意力一致性 （TAVAC），这是一种用于评估 ViT 模型过拟合和量化解释可重复性的指标。通过比较训练和测试之间的高关注区域，我们在四个公共图像分类数据集和两个独立的乳腺癌组织学图像数据集上测试了 TAVAC。过拟合模型显示 TAVAC 评分显著降低。TAVAC 还可以区分脱靶关注和靶向关注，并在细粒度细胞水平上测量解释泛化。除了诊断之外，TAVAC 还增强了基础研究中的解释可重复性，揭示了生物医学和其他一般非生物医学图像的关键空间模式和细胞结构。