Multimodal remote sensing image registration is crucial for multimodal information fusion and applications. The significant nonlinear appearance difference between multimodal images caused by the various imaging mechanisms dramatically increases the challenge of image registration. This article proposes an adaptive frequency feature filtering network (F3Net) for cross-modal remote sensing image registration. On the one hand, F3Net explicitly explores the useful frequency components across modal images based on multilevel deep features. On the other hand, F3Net can take advantage of the nonlocal receptive fields by frequency modulation for feature learning and boosting image registration performances. F3Net inserts frequency feature filtering (F3) modules in multilevel deep features. Specifically, F3Net first performs the fast Fourier transform (FFT) for deep features. Then, F3Net designs a frequency attention (FA) module to adaptive enhance the shared and discriminative frequency features between multimodal images while suppressing the frequency components that hinder the cross-modal image registration. In addition, F3Net adopts multiscale frequency filtering fusion to facilitate discriminative feature learning, including global frequency feature filtering (GF3) based on the global image spectrum and local frequency feature filtering (LF3) based on the spectrum of stacked image regions. Experimental results on many remote sensing images have demonstrated the efficiency of the F3Net on multimodal image registration.

中文翻译：

---

F3Net：用于多模态遥感图像配准的自适应频率特征过滤网络


多模态遥感图像配准对于多模态信息融合和应用至关重要。由各种成像机制引起的多模态图像之间显着的非线性外观差异极大地增加了图像配准的挑战。本文提出了一种用于跨模态遥感图像配准的自适应频率特征过滤网络（F3Net）。一方面，F3Net 基于多级深度特征显式探索模态图像中的有用频率分量。另一方面，F3Net 可以通过频率调制来利用非局部感受野进行特征学习并提高图像配准性能。 F3Net 在多级深度特征中插入频率特征过滤（F3）模块。具体来说，F3Net 首先对深层特征执行快速傅里叶变换（FFT）。然后，F3Net 设计了频率注意（FA）模块来自适应增强多模态图像之间的共享和判别性频率特征，同时抑制阻碍跨模态图像配准的频率分量。此外，F3Net采用多尺度频率滤波融合来促进判别性特征学习，包括基于全局图像频谱的全局频率特征滤波（GF3）和基于堆叠图像区域频谱的局部频率特征滤波（LF3）。许多遥感图像的实验结果证明了F3Net在多模态图像配准方面的效率。