Multispectral remote sensing image-based sea fog detection (SFD) is both important and challenging. Deep learning methods for SFD have become mainstream due to their powerful nonlinear learning capabilities and flexibility. However, existing methods have not fully utilized the physical difference priors in multispectral images (MSI) for SFD, making it difficult to skillfully capture the shape and appearance characteristics of sea fog, thus leading to uncertainties in SFD. We propose the spatial–spectral difference network (S2DNet), a deep encoding–decoding framework that merges inter-spectral and intra-spectral heterogeneous difference features. Specifically, inspired by physics-based difference threshold methods, we developed a physics-inspired inter-spectral difference module (PIDM) that combines feature-level difference with deep neural networks to capture the shape characteristics of sea fog. We designed the intra-spectral difference module (ISDM) using difference convolution to represent sea fog’s fine-grained and dynamic appearance information. Furthermore, inspired by multi-view learning, we propose heterogeneous feature mutual learning (HFML) that seeks robust representations by focusing on semantically invariant aspects within heterogeneous difference features, adapting to the dynamic nature of sea fog. HFML is achieved through global feature mutual learning using an adversarial procedure and local feature mutual learning supported by a novel information-theoretic objective function that links maximizing statistical correlation with expectation maximization. Experiments on two SFD datasets show that integrating physical difference priors into deep learning improves SFD. In both continuous temporal and high spatial resolution SFD tasks, S2DNet outperforms existing advanced deep learning methods. Moreover, S2DNet demonstrates stronger robustness under degraded remote sensing image conditions, highlighting its potential usefulness and practicality in real-world applications.

中文翻译：

---

具有异构特征互学习的深度时空-光谱差分网络用于海雾检测


基于图像的多光谱遥感海雾检测 （SFD） 既重要又具有挑战性。SFD 的深度学习方法由于其强大的非线性学习能力和灵活性而成为主流。然而，现有方法并未充分利用多光谱图像 （MSI） 中用于 SFD 的物理差异，难以巧妙捕捉海雾的形状和外观特征，从而导致 SFD 的不确定性。我们提出了空间-光谱差异网络 （S2DNet），这是一个深度编码-解码框架，它融合了光谱间和光谱内异构差异特征。具体来说，受基于物理的差分阈值方法的启发，我们开发了一种受物理启发的光谱间差分模块 （PIDM），它将特征级差分与深度神经网络相结合，以捕获海雾的形状特征。我们使用差分卷积设计了光谱内差分模块 （ISDM） 来表示海雾的细粒度和动态外观信息。此外，受多视图学习的启发，我们提出了异构特征互学习 （HFML），它通过关注异构差异特征中的语义不变方面来寻求稳健的表示，以适应海雾的动态性质。HFML 是通过使用对抗程序的全局特征互学习和局部特征互学习实现的，该互学习由新颖的信息论目标函数支持，该目标函数将最大化统计相关性与期望最大化联系起来。在两个 SFD 数据集上的实验表明，将物理差异先验集成到深度学习中可以提高 SFD 。 在连续时间和高空间分辨率 SFD 任务中，S2DNet 的性能优于现有的高级深度学习方法。此外，S2DNet 在退化的遥感图像条件下表现出更强的鲁棒性，凸显了其在实际应用中的潜在有用性和实用性。