Automatic analysis of colonoscopy images has been an active field of research motivated by the importance of early detection of precancerous polyps. However, detecting polyps during the live examination can be challenging due to various factors such as variation of skills and experience among the endoscopists, lack of attentiveness, and fatigue leading to a high polyp miss-rate. Therefore, there is a need for an automated system that can flag missed polyps during the examination and improve patient care. Deep learning has emerged as a promising solution to this challenge as it can assist endoscopists in detecting and classifying overlooked polyps and abnormalities in real time, improving the accuracy of diagnosis and enhancing treatment. In addition to the algorithm’s accuracy, transparency and interpretability are crucial to explaining the whys and hows of the algorithm’s prediction. Further, conclusions based on incorrect decisions may be fatal, especially in medicine. Despite these pitfalls, most algorithms are developed in private data, closed source, or proprietary software, and methods lack reproducibility. Therefore, to promote the development of efficient and transparent methods, we have organized the “Medico automatic polyp segmentation (Medico 2020)” and “MedAI: Transparency in Medical Image Segmentation (MedAI 2021)” competitions. The Medico 2020 challenge received submissions from 17 teams, while the MedAI 2021 challenge also gathered submissions from another 17 distinct teams in the following year. We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic. Our analysis revealed that the participants improved dice coefficient metrics from 0.8607 in 2020 to 0.8993 in 2021 despite adding diverse and challenging frames (containing irregular, smaller, sessile, or flat polyps), which are frequently missed during a routine clinical examination. For the instrument segmentation task, the best team obtained a mean Intersection over union metric of 0.9364. For the transparency task, a multi-disciplinary team, including expert gastroenterologists, accessed each submission and evaluated the team based on open-source practices, failure case analysis, ablation studies, usability and understandability of evaluations to gain a deeper understanding of the models’ credibility for clinical deployment. The best team obtained a final transparency score of 21 out of 25. Through the comprehensive analysis of the challenge, we not only highlight the advancements in polyp and surgical instrument segmentation but also encourage subjective evaluation for building more transparent and understandable AI-based colonoscopy systems. Moreover, we discuss the need for multi-center and out-of-distribution testing to address the current limitations of the methods to reduce the cancer burden and improve patient care.

中文翻译：

---

通过 Medico 2020 和 MedAI 2021 挑战验证结肠镜检查中的息肉和仪器分割方法


由于早期检测癌前息肉的重要性，结肠镜检查图像的自动分析一直是一个活跃的研究领域。然而，由于各种因素，例如内窥镜医师的技能和经验差异、注意力不集中以及疲劳导致息肉漏检率较高，在实时检查期间检测息肉可能具有挑战性。因此，需要一种自动化系统，可以在检查过程中标记漏检的息肉并改善患者护理。深度学习已成为应对这一挑战的一种有前途的解决方案，因为它可以帮助内窥镜医生实时检测和分类被忽视的息肉和异常，从而提高诊断的准确性并加强治疗。除了算法的准确性之外，透明度和可解释性对于解释算法预测的原因和方式也至关重要。此外，基于错误决策的结论可能是致命的，尤其是在医学领域。尽管存在这些缺陷，大多数算法都是在私有数据、闭源或专有软件中开发的，并且方法缺乏可重复性。因此，为了促进高效、透明方法的发展，我们组织了“Medico自动息肉分割（Medico 2020）”和“MedAI：医学图像分割的透明度（MedAI 2021）”竞赛。 Medico 2020 挑战赛收到了 17 个团队的参赛作品，而 MedAI 2021 挑战赛也在次年收集了另外 17 个不同团队的参赛作品。我们对每项贡献进行了全面的总结和分析，强调了表现最佳的方法的优势，并讨论了将此类方法转化为临床的可能性。 我们的分析显示，尽管添加了多样化且具有挑战性的框架（包含不规则、较小、无蒂或扁平息肉），而这些在常规临床检查中经常被遗漏，但参与者的骰子系数指标从 2020 年的 0.8607 提高到 2021 年的 0.8993。对于仪器分割任务，最好的团队获得了 0.9364 的平均交集度量。对于透明度任务，包括胃肠病专家在内的多学科团队访问了每份提交内容，并根据开源实践、失败案例分析、消融研究、评估的可用性和可理解性对团队进行了评估，以更深入地了解模型的临床部署的可信度。最佳团队最终获得了 21 分（满分 25 分）的透明度分数。通过对挑战的综合分析，我们不仅强调了息肉和手术器械分割方面的进展，还鼓励主观评估，以构建更透明、更易于理解的基于人工智能的结肠镜检查系统。此外，我们讨论了多中心和分布外检测的必要性，以解决当前减轻癌症负担和改善患者护理方法的局限性。