Colorectal liver metastases (CLM) affect almost half of all colon cancer patients and the response to systemic chemotherapy plays a crucial role in patient survival. While oncologists typically use tumor grading scores, such as tumor regression grade (TRG), to establish an accurate prognosis on patient outcomes, including overall survival (OS) and time-to-recurrence (TTR), these traditional methods have several limitations. They are subjective, time-consuming, and require extensive expertise, which limits their scalability and reliability. Additionally, existing approaches for prognosis prediction using machine learning mostly rely on radiological imaging data, but recently histological images have been shown to be relevant for survival predictions by allowing to fully capture the complex microenvironmental and cellular characteristics of the tumor. To address these limitations, we propose an end-to-end approach for automated prognosis prediction using histology slides stained with Hematoxylin and Eosin (H&E) and Hematoxylin Phloxine Saffron (HPS). We first employ a Generative Adversarial Network (GAN) for slide normalization to reduce staining variations and improve the overall quality of the images that are used as input to our prediction pipeline. We propose a semi-supervised model to perform tissue classification from sparse annotations, producing segmentation and feature maps. Specifically, we use an attention-based approach that weighs the importance of different slide regions in producing the final classification results. Finally, we exploit the extracted features for the metastatic nodules and surrounding tissue to train a prognosis model. In parallel, we train a vision Transformer model in a knowledge distillation framework to replicate and enhance the performance of the prognosis prediction. We evaluate our approach on an in-house clinical dataset of 258 CLM patients, achieving superior performance compared to other comparative models with a c-index of 0.804 (0.014) for OS and 0.735 (0.016) for TTR, as well as on two public datasets. The proposed approach achieves an accuracy of 86.9% to 90.3% in predicting TRG dichotomization. For the 3-class TRG classification task, the proposed approach yields an accuracy of 78.5% to 82.1%, outperforming the comparative methods. Our proposed pipeline can provide automated prognosis for pathologists and oncologists, and can greatly promote precision medicine progress in managing CLM patients.

中文翻译：

---

半监督 ViT 知识蒸馏网络，具有风格转移归一化功能，用于结直肠肝转移生存预测


结直肠肝转移 （CLM） 影响了近一半的结肠癌患者，对全身化疗的反应对患者的生存起着至关重要的作用。虽然肿瘤学家通常使用肿瘤分级评分，例如肿瘤消退分级 （TRG），来确定患者结局的准确预后，包括总生存期 （OS） 和复发时间 （TTR），但这些传统方法有几个局限性。它们是主观的、耗时的，并且需要广泛的专业知识，这限制了它们的可扩展性和可靠性。此外，使用机器学习的现有预后预测方法主要依赖于放射成像数据，但最近组织学图像已被证明与生存预测相关，因为它允许充分捕捉肿瘤的复杂微环境和细胞特征。为了解决这些限制，我们提出了一种端到端方法，使用苏木精和伊红 （H&E） 和苏木精 Phloxine Saffron （HPS） 染色的组织学载玻片进行自动预后预测。我们首先采用生成对抗网络 （GAN） 进行载玻片归一化，以减少染色变化并提高用作预测管道输入的图像的整体质量。我们提出了一个半监督模型来从稀疏注释中执行组织分类，生成分割和特征图。具体来说，我们使用一种基于注意力的方法，权衡不同幻灯片区域在产生最终分类结果中的重要性。最后，我们利用提取的转移性结节和周围组织的特征来训练预后模型。 同时，我们在知识蒸馏框架中训练一个视觉 Transformer 模型，以复制和增强预后预测的性能。我们在 258 名 CLM 患者的内部临床数据集上评估了我们的方法，与其他比较模型相比，OS 的 c 指数为 0.804 （0.014），TTR 的 c 指数为 0.735 （0.016），以及两个公共数据集。所提出的方法在预测 TRG 二分法方面实现了 86.9% 到 90.3% 的准确率。对于 3 类 TRG 分类任务，所提出的方法的准确率为 78.5% 至 82.1%，优于比较方法。我们提出的管道可以为病理学家和肿瘤学家提供自动化预后，并且可以极大地促进管理 CLM 患者的精准医学进展。