Breast cancer is a significant global public health concern, with various treatment options available based on tumor characteristics. Pathological examination of excision specimens after surgery provides essential information for treatment decisions. However, the manual selection of representative sections for histological examination is laborious and subjective, leading to potential sampling errors and variability, especially in carcinomas that have been previously treated with chemotherapy. Furthermore, the accurate identification of residual tumors presents significant challenges, emphasizing the need for systematic or assisted methods to address this issue. In order to enable the development of deep-learning algorithms for automated cancer detection on radiology images, it is crucial to perform radiology-pathology registration, which ensures the generation of accurately labeled ground truth data. The alignment of radiology and histopathology images plays a critical role in establishing reliable cancer labels for training deep-learning algorithms on radiology images. However, aligning these images is challenging due to their content and resolution differences, tissue deformation, artifacts, and imprecise correspondence. We present a novel deep learning-based pipeline for the affine registration of faxitron images, the x-ray representations of macrosections of ex-vivo breast tissue, and their corresponding histopathology images of tissue segments. The proposed model combines convolutional neural networks and vision transformers, allowing it to effectively capture both local and global information from the entire tissue macrosection as well as its segments. This integrated approach enables simultaneous registration and stitching of image segments, facilitating segment-to-macrosection registration through a puzzling-based mechanism. To address the limitations of multi-modal ground truth data, we tackle the problem by training the model using synthetic mono-modal data in a weakly supervised manner. The trained model demonstrated successful performance in multi-modal registration, yielding registration results with an average landmark error of 1.51 mm (±2.40), and stitching distance of 1.15 mm (±0.94). The results indicate that the model performs significantly better than existing baselines, including both deep learning-based and iterative models, and it is also approximately 200 times faster than the iterative approach. This work bridges the gap in the current research and clinical workflow and has the potential to improve efficiency and accuracy in breast cancer evaluation and streamline pathology workflow.

中文翻译：

---

PViT-AIR：用于乳腺组织多组织病理学和传真管图像的令人费解的基于视觉转换器的仿射图像配准


乳腺癌是一个重要的全球公共卫生问题，根据肿瘤特征有多种治疗方案可供选择。手术后切除标本的病理检查为治疗决策提供了必要的信息。然而，手动选择代表性切片进行组织学检查既费力又主观，导致潜在的取样误差和可变性，尤其是在既往接受过化疗的癌症中。此外，准确识别残留肿瘤带来了重大挑战，强调需要系统或辅助方法来解决这个问题。为了能够开发用于放射学图像上自动癌症检测的深度学习算法，执行放射学-病理学配准至关重要，这可确保生成准确标记的真值数据。放射学和组织病理学图像的对齐在建立可靠的癌症标签以在放射学图像上训练深度学习算法方面起着关键作用。然而，由于这些图像的内容和分辨率差异、组织变形、伪影和不精确的对应关系，对齐这些图像具有挑战性。我们提出了一种基于深度学习的新型管道，用于仿射配准 faxitron 图像、离体乳腺组织宏观切片的 X 射线表示以及组织节段的相应组织病理学图像。所提出的模型结合了卷积神经网络和视觉转换器，使其能够有效地从整个组织宏观切片及其节段中捕获局部和全局信息。 这种集成方法可以同时对图像片段进行配准和拼接，从而通过基于谜题的机制促进片段到宏切片的配准。为了解决多模态真值数据的局限性，我们通过使用合成单模态数据以弱监督方式训练模型来解决这个问题。经过训练的模型在多模态套准方面表现出了成功的性能，产生了平均地标误差为 1.51 毫米 （±2.40） 和拼接距离为 1.15 毫米 （±0.94） 的套准结果。结果表明，该模型的性能明显优于现有基线，包括基于深度学习的模型和迭代模型，并且它也比迭代方法快约 200 倍。这项工作弥合了当前研究和临床工作流程中的差距，有可能提高乳腺癌评估的效率和准确性并简化病理学工作流程。