Stereo matching of satellite images presents challenges due to missing data, domain differences, and imperfect rectification. To address these issues, we propose cascaded recurrent networks with masked representation learning for high-resolution satellite stereo images, consisting of feature extraction and cascaded recurrent modules. First, we develop the correlation computation in the cascaded recurrent module to search for results on the epipolar line and adjacent areas, mitigating the impacts of erroneous rectification. Second, we use a training strategy based on masked representation learning to handle missing data and different domain attributes, enhancing data utilization and feature representation. Our training strategy includes two stages: (1) image reconstruction stage. We feed masked left or right images to the feature extraction module and adopt a reconstruction decoder to reconstruct the original images as a pre-training process, obtaining a pre-trained feature extraction module; (2) the stereo matching stage. We lock the parameters of the feature extraction module and employ stereo image pairs to train the cascaded recurrent module to get the final model. We implement the cascaded recurrent networks with two well-known feature extraction modules (CNN-based Restormer or Transformer-based ViT) to prove the effectiveness of our approach. Experimental results on the US3D and WHU-Stereo datasets show that: (1) Our training strategy can be used for CNN-based and Transformer-based methods on the remote sensing datasets with limited data to improve performance, outperforming the second-best network HMSM-Net by approximately 0.54% and 1.95% in terms of the percentage of the 3-px error on the WHU-Stereo and US3D datasets, respectively; (2) Our correlation manner can handle imperfect rectification, reducing the error rate by 8.9% on the random shift test; (3) Our method can predict high-quality disparity maps and achieve state-of-the-art performance, reducing the percentage of the 3-px error to 12.87% and 7.01% on the WHU-Stereo and US3D datasets, respectively. The source codes are released at https://github.com/Archaic-Atom/MaskCRNet.

中文翻译：

---

具有掩蔽表示学习的级联循环网络，用于高分辨率卫星图像的立体匹配


由于数据缺失、域差异和不完全校正，卫星图像的立体匹配带来了挑战。为了解决这些问题，我们提出了用于高分辨率卫星立体图像的具有掩蔽表示学习的级联递归网络，由特征提取和级联递归模块组成。首先，我们在级联循环模块中开发相关计算，以搜索外极线和相邻区域的结果，减轻错误整流的影响。其次，我们使用基于掩码表示学习的训练策略来处理缺失数据和不同的域属性，从而提高数据利用率和特征表示。我们的训练策略包括两个阶段：（1） 图像重建阶段。我们将被遮罩的左侧或右侧图像馈送到特征提取模块，并采用重建解码器对原始图像进行重建作为预训练过程，得到一个预训练的特征提取模块;（2） 立体声匹配阶段。我们锁定特征提取模块的参数，并使用立体图像对来训练级联递归模块以获得最终模型。我们使用两个众所周知的特征提取模块（基于 CNN 的 Restormer 或基于 Transformer 的 ViT）来实现级联循环网络，以证明我们方法的有效性。在 US3D 和 WHU-Stereo 数据集上的实验结果表明：（1） 我们的训练策略可用于数据有限的遥感数据集上基于 CNN 和基于 Transformer 的方法，以提高性能，性能比第二好的网络 HMSM-Net 高出约 0.54% 和 1.在 WHU-Stereo 和 US3D 数据集上的 3 像素误差百分比方面分别为 95%;（2） 我们的相关方式可以处理不完美的整流，在随机移位测试中降低 8.9% 的错误率;（3） 我们的方法可以预测高质量的视差图并实现最先进的性能，在 WHU-Stereo 和 US3D 数据集上将 3 px 误差的百分比分别降低到 12.87% 和 7.01%。源代码在 https://github.com/Archaic-Atom/MaskCRNet 发布。