Accurate and automatic segmentation of medical images plays an essential role in clinical diagnosis and analysis. It has been established that integrating contextual relationships substantially enhances the representational ability of neural networks. Conventionally, Long Short-Term Memory (LSTM) and Self-Attention (SA) mechanisms have been recognized for their proficiency in capturing global dependencies within data. However, these mechanisms have typically been viewed as distinct modules without a direct linkage. This paper presents the integration of LSTM design with SA sparse coding as a key innovation. It uses linear combinations of LSTM states for SA’s query, key, and value (QKV) matrices to leverage LSTM’s capability for state compression and historical data retention. This approach aims to rectify the shortcomings of conventional sparse coding methods that overlook temporal information, thereby enhancing SA’s ability to do sparse coding and capture global dependencies. Building upon this premise, we introduce two innovative modules that weave the SA matrix into the LSTM state design in distinct manners, enabling LSTM to more adeptly model global dependencies and meld seamlessly with SA without accruing extra computational demands. Both modules are separately embedded into the U-shaped convolutional neural network architecture for handling both 2D and 3D medical images. Experimental evaluations on downstream medical image segmentation tasks reveal that our proposed modules not only excel on four extensively utilized datasets across various baselines but also enhance prediction accuracy, even on baselines that have already incorporated contextual modules. Code is available at https://github.com/yeshunlong/SALSTM .

中文翻译：

---

稀疏编码启发了用于医学图像分割的 LSTM 和自注意力集成


准确、自动分割医学影像在临床诊断和分析中起着至关重要的作用。已经确定，整合上下文关系可以大大增强神经网络的表示能力。传统上，长短期记忆 （LSTM） 和自我注意 （SA） 机制因其在捕获数据中的全局依赖关系方面的熟练程度而受到认可。然而，这些机制通常被视为没有直接联系的不同模块。本文将 LSTM 设计与 SA 稀疏编码的集成作为一项关键创新。它将 LSTM 状态的线性组合用于 SA 的查询、键和值 （QKV） 矩阵，以利用 LSTM 的状态压缩和历史数据保留功能。这种方法旨在纠正传统稀疏编码方法忽略时间信息的缺点，从而增强 SA 进行稀疏编码和捕获全局依赖关系的能力。基于这个前提，我们引入了两个创新模块，它们以不同的方式将 SA 矩阵编织到 LSTM 状态设计中，使 LSTM 能够更熟练地对全局依赖关系进行建模，并与 SA 无缝融合，而不会产生额外的计算需求。这两个模块分别嵌入到 U 形卷积神经网络架构中，用于处理 2D 和 3D 医学图像。对下游医学图像分割任务的实验评估表明，我们提出的模块不仅在各种基线的四个广泛使用的数据集上表现出色，而且还提高了预测准确性，即使在已经包含上下文模块的基线上也是如此。代码可在 https://github.com/yeshunlong/SALSTM 获取。