The human epidermal growth factor receptor 2 (HER2) gene amplification status is a crucial marker for evaluating clinical therapies of breast or gastric cancer. We propose a deep learning-based pipeline for the detection, localization and classification of interphase nuclei depending on their HER2 gene amplification state in Fluorescence in situ hybridization (FISH) images. Our pipeline combines two RetinaNet-based object localization networks which are trained (1) to detect and classify interphase nuclei into distinct classes normal, low-grade and high-grade and (2) to detect and classify FISH signals into distinct classes HER2 or centromere of chromosome 17 (CEN17). By independently classifying each nucleus twice, the two-step pipeline provides both robustness and interpretability for the automated detection of the HER2 amplification status. The accuracy of our deep learning-based pipeline is on par with that of three pathologists and a set of 57 validation images containing several hundreds of nuclei are accurately classified. The automatic pipeline is a first step towards assisting pathologists in evaluating the HER2 status of tumors using FISH images, for analyzing FISH images in retrospective studies, and for optimizing the documentation of each tumor sample by automatically annotating and reporting of the HER2 gene amplification specificities.

人类表皮生长因子受体2（HER2）基因的扩增状态是评估乳腺癌或胃癌临床疗法的重要标志。我们提出了一个基于深度学习的管道，用于检测，定位和分类相间核，具体取决于它们在荧光原位杂交（FISH）图像中的HER2基因扩增状态。我们的管道结合了两个基于RetinaNet的对象定位网络，这些网络经过训练（1）将相间核检测和分类为正常，低级和高级的不同类别，以及（2）将FISH信号检测和分类为不同的类别HER2或着丝粒17号染色​​体（CEN17）。通过将每个原子核独立分类两次，两步流水线为自动检测HER2扩增状态提供了鲁棒性和可解释性。我们基于深度学习的管道的准确性与三位病理学家的准确性相当，并且对包含数百个细胞核的57幅验证图像进行了准确分类。自动流水线是协助病理学家使用FISH图像评估肿瘤HER2状态，在回顾性研究中分析FISH图像以及通过自动注释和报告HER2基因扩增特异性来优化每个肿瘤样本文档的第一步。