Cancer is widely recognized as the primary cause of mortality worldwide, and pathology analysis plays a pivotal role in achieving accurate cancer diagnosis. The intricate representation of features in histopathological images encompasses abundant information crucial for disease diagnosis, regarding cell appearance, tumor microenvironment, and geometric characteristics. However, recent deep learning methods have not adequately exploited geometric features for pathological image classification due to the absence of effective descriptors that can capture both cell distribution and gathering patterns, which often serve as potent indicators. In this paper, inspired by clinical practice, a Hierarchical Graph Pyramid Transformer (HGPT) is proposed to guide pathological image classification by effectively exploiting a geometric representation of tissue distribution which was ignored by existing state-of-the-art methods. First, a graph representation is constructed according to morphological feature of input pathological image and learn geometric representation through the proposed multi-head graph aggregator. Then, the image and its graph representation are feed into the transformer encoder layer to model long-range dependency. Finally, a locality feature enhancement block is designed to enhance the 2D local representation of feature embedding, which is not well explored in the existing vision transformers. An extensive experimental study is conducted on Kather-5K, MHIST, NCT-CRC-HE, and GasHisSDB for binary or multi-category classification of multiple cancer types. Results demonstrated that our method is capable of consistently reaching superior classification outcomes for histopathological images, which provide an effective diagnostic tool for malignant tumors in clinical practice.

中文翻译：

---

通过分层图金字塔变压器利用几何特征使用组织病理学图像进行癌症诊断


癌症被广泛认为是全世界死亡的主要原因，病理分析在实现准确的癌症诊断中发挥着关键作用。组织病理学图像中特征的复杂表示包含对疾病诊断至关重要的丰富信息，包括细胞外观、肿瘤微环境和几何特征。然而，由于缺乏可以捕获细胞分布和聚集模式的有效描述符，而这些描述符通常可以作为有效的指标，因此最近的深度学习方法尚未充分利用几何特征进行病理图像分类。在本文中，受临床实践的启发，提出了一种层次图金字塔变换器（HGPT），通过有效利用现有最先进方法忽略的组织分布的几何表示来指导病理图像分类。首先，根据输入病理图像的形态特征构建图表示，并通过所提出的多头图聚合器学习几何表示。然后，图像及其图形表示被输入到变压器编码器层以模拟远程依赖性。最后，设计了局部特征增强块来增强特征嵌入的二维局部表示，这在现有的视觉变换器中没有得到很好的探索。在 Kather-5K、MHIST、NCT-CRC-HE 和 GasHisSDB 上进行了广泛的实验研究，用于多种癌症类型的二元或多类别分类。 结果表明，我们的方法能够始终如一地达到组织病理学图像的优异分类结果，为临床实践中的恶性肿瘤提供有效的诊断工具。