<br>CRS-Diff：利用扩散模型生成可控遥感图像,IEEE Transactions on Geoscience and Remote Sensing

当前位置： X-MOL 学术 › IEEE Trans. Geosci. Remote Sens. › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

CRS-Diff：利用扩散模型生成可控遥感图像
IEEE Transactions on Geoscience and Remote Sensing ( IF 7.5 ) Pub Date : 2024-09-02 , DOI: 10.1109/tgrs.2024.3453414
Datao Tang ₁ , Xiangyong Cao ₁ , Xingsong Hou ₂ , Zhongyuan Jiang ₃ , Junmin Liu ₄ , Deyu Meng ₄

Affiliation

生成模型的出现彻底改变了遥感（RS）图像生成领域。尽管可以生成高质量的图像，但现有方法主要依赖于文本控制条件，因此并不总是能够准确稳定地生成图像。在本文中，我们提出了 CRS-Diff，这是一种专门为 RS 图像生成量身定制的新型 RS 生成框架，利用扩散模型的固有优势，同时集成更先进的控制机制。具体来说，CRS-Diff可以同时支持文本条件、元数据条件和图像条件控制输入，从而实现更精确的控制以细化生成过程。为了有效地整合多种条件控制信息，我们引入了一种新的条件控制机制来实现多尺度特征融合（FF），从而增强了控制条件的指导效果。据我们所知，CRS-Diff 是第一个多条件可控 RS 生成模型。单条件和多条件情况下的实验结果表明，与之前的方法相比，我们的 CRS-Diff 在定量和定性生成 RS 图像方面具有优越的能力。此外，我们的 CRS-Diff 可以作为数据引擎，为下游任务（例如道路提取）生成高质量的训练数据。该代码可在 https://github.com/Sonettoo/CRS-Diff 获取。

"点击查看英文标题和摘要"

CRS-Diff: Controllable Remote Sensing Image Generation With Diffusion Model

The emergence of generative models has revolutionized the field of remote sensing (RS) image generation. Despite generating high-quality images, existing methods are limited in relying mainly on text control conditions, and thus do not always generate images accurately and stably. In this article, we propose CRS-Diff, a new RS generative framework specifically tailored for RS image generation, leveraging the inherent advantages of diffusion models while integrating more advanced control mechanisms. Specifically, CRS-Diff can simultaneously support text-condition, metadata-condition, and image-condition control inputs, thus enabling more precise control to refine the generation process. To effectively integrate multiple condition control information, we introduce a new conditional control mechanism to achieve multiscale feature fusion (FF), thus enhancing the guiding effect of control conditions. To the best of our knowledge, CRS-Diff is the first multiple-condition controllable RS generative model. Experimental results in single-condition and multiple-condition cases have demonstrated the superior ability of our CRS-Diff to generate RS images both quantitatively and qualitatively compared with previous methods. Additionally, our CRS-Diff can serve as a data engine that generates high-quality training data for downstream tasks, e.g., road extraction. The code is available at https://github.com/Sonettoo/CRS-Diff .

更新日期：2024-09-02

点击分享查看原文

点击收藏

阅读更多本刊新发论文本刊介绍/投稿指南