当前位置: X-MOL 学术ISPRS J. Photogramm. Remote Sens. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Semantic guided large scale factor remote sensing image super-resolution with generative diffusion prior
ISPRS Journal of Photogrammetry and Remote Sensing ( IF 10.6 ) Pub Date : 2024-12-14 , DOI: 10.1016/j.isprsjprs.2024.12.001
Ce Wang, Wanjie Sun

In the realm of remote sensing, images captured by different platforms exhibit significant disparities in spatial resolution. Consequently, effective large scale factor super-resolution (SR) algorithms are vital for maximizing the utilization of low-resolution (LR) satellite data captured from orbit. However, existing methods confront challenges such as semantic inaccuracies and blurry textures in the reconstructed images. To tackle these issues, we introduce a novel framework, the Semantic Guided Diffusion Model (SGDM), designed for large scale factor remote sensing image super-resolution. The framework exploits a pre-trained generative model as a prior to generate perceptually plausible high-resolution (HR) images, thereby constraining the solution space and mitigating texture blurriness. We further enhance the reconstruction by incorporating vector maps, which carry structural and semantic cues to enhance the reconstruction fidelity of ground objects. Moreover, pixel-level inconsistencies in paired remote sensing images, stemming from sensor-specific imaging characteristics, may hinder the convergence of the model and the diversity in generated results. To address this problem, we develop a method to extract sensor-specific imaging characteristics and model the distribution of them. The proposed model can decouple imaging characteristics from image content, allowing it to generate diverse super-resolution images based on imaging characteristics provided by reference satellite images or sampled from the imaging characteristic probability distributions. To validate and evaluate our approach, we create the Cross-Modal Super-Resolution Dataset (CMSRD). Qualitative and quantitative experiments on CMSRD showcase the superiority and broad applicability of our method. Experimental results on downstream vision tasks also demonstrate the utilitarian of the generated SR images. The dataset and code will be publicly available at https://github.com/wwangcece/SGDM.

中文翻译:


具有生成扩散先验的语义引导大比例因子遥感图像超分辨率



在遥感领域,不同平台捕获的图像在空间分辨率上表现出显着差异。因此,有效的大比例因子超分辨率 (SR) 算法对于最大限度地利用从轨道捕获的低分辨率 (LR) 卫星数据至关重要。然而,现有方法面临着诸如语义不准确和重建图像中纹理模糊等挑战。为了解决这些问题,我们引入了一种新的框架,即语义引导扩散模型 (SGDM),专为大比例尺遥感图像超分辨率而设计。该框架利用预先训练的生成模型作为先生成感知合理的高分辨率 (HR) 图像,从而限制解决方案空间并减轻纹理模糊。我们通过结合矢量地图进一步增强重建,矢量地图带有结构和语义线索,以提高地面物体的重建保真度。此外,由于传感器特定的成像特性,配对遥感图像中的像素级不一致可能会阻碍模型的收敛和生成结果的多样性。为了解决这个问题,我们开发了一种方法来提取传感器特定的成像特征并对其分布进行建模。所提出的模型可以将成像特征与图像内容解耦,使其能够根据参考卫星图像提供的成像特征或从成像特征概率分布中采样的成像特征生成多样化的超分辨率图像。为了验证和评估我们的方法,我们创建了跨模态超分辨率数据集 (CMSRD)。CMSRD 的定性和定量实验展示了我们方法的优越性和广泛适用性。 下游视觉任务的实验结果也证明了生成的 SR 图像的实用性。数据集和代码将在 https://github.com/wwangcece/SGDM 上公开提供。
更新日期:2024-12-14
down
wechat
bug