A significant challenge of spectral reconstruction (SR) task is the lower performance reconstructed in foreground regions compared to background regions, which can be attributed to the marked difference in diversity of objects and disparity of adjacent scene characteristics. Moreover, the reconstruction of edge regions is often fraught with substantial errors due to the transitional nature of these regions, an issue conventional single convolutional neural networks (CNNs) and transformers struggle to handle. To address these challenges, we introduce the residual mask in cascaded convolutional transformer (RC2T) to iteratively improve the reconstruction of hyperspectral images (HSIs). Specifically, we propose a residual-predict mask generator (RMG) to generate a residual mask that retains band properties to separate feature with different complexities. Meanwhile, to achieve band expansion of mask features within the autoencoder, we approximate it to a Markov process and exploit the multistage spectral-aware Markov transfer (MMT) for its lightweight implementation. Next, we introduce the parallel convolutional multihead self-attention module (PSM), in which CNN runs parallel to the transformer to handle simple and complex features separately. Additionally, the residual mask loss function uses the established relationship between complexity of feature and reconstruction accuracy to generate residual mask in a self-supervised manner for providing complex high-frequency prior. We have validated our approach using three published datasets (NTIRE 2020 “Clean” track, NTIRE 2022, and CAVE). Additionally, we also conducted experiments with the proposed method on remote sensing dataset grss_dfc_2018 and a satellite-borne remote sensing dataset, achieving optimal performance. The experimental results demonstrate that our RC2T method is state-of-the-art (SOTA) in the field of SR.

中文翻译：

---

用于谱重建的级联卷积变压器中的残余掩模


光谱重建（SR）任务的一个重大挑战是前景区域重建的性能低于背景区域，这可归因于对象多样性的显着差异和相邻场景特征的差异。此外，由于边缘区域的过渡性质，边缘区域的重建常常充满重大错误，这是传统的单卷积神经网络（CNN）和变压器难以处理的问题。为了解决这些挑战，我们在级联卷积变换器（RC2T）中引入残差掩模来迭代改进高光谱图像（HSIs）的重建。具体来说，我们提出了一种残差预测掩模生成器（RMG）来生成残差掩模，该残差掩模保留带属性以分离具有不同复杂性的特征。同时，为了实现自动编码器内掩模特征的频带扩展，我们将其近似为马尔可夫过程，并利用多级频谱感知马尔可夫传输（MMT）进行轻量级实现。接下来，我们介绍并行卷积多头自注意力模块（PSM），其中 CNN 与 Transformer 并行运行，以分别处理简单和复杂的特征。此外，残余掩模损失函数利用特征复杂性和重建精度之间已建立的关系以自监督方式生成残余掩模，以提供复杂的高频先验。我们使用三个已发布的数据集（NTIRE 2020“Clean”track、NTIRE 2022 和 CAVE）验证了我们的方法。此外，我们还在遥感数据集grss_dfc_2018和星载遥感数据集上进行了实验，取得了最佳性能。 实验结果表明，我们的 RC2T 方法在 SR 领域是最先进的（SOTA）。