当前位置: X-MOL 学术IEEE Trans. Image Process. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Explicitly-Decoupled Text Transfer With Minimized Background Reconstruction for Scene Text Editing
IEEE Transactions on Image Processing ( IF 10.8 ) Pub Date : 2024-10-15 , DOI: 10.1109/tip.2024.3477355
Jianqun Zhou, Pengwen Dai, Yang Li, Manjiang Hu, Xiaochun Cao

Scene text editing aims to replace the source text with the target text while preserving the original background. Its practical applications span various domains, such as data generation and privacy protection, highlighting its increasing importance in recent years. In this study, we propose a novel Scene Text Editing network with Explicitly-decoupled text transfer and Minimized background reconstruction, called STEEM. Unlike existing methods that usually fuse text style, text content, and background, our approach focuses on decoupling text style and content from the background and utilizes the minimized background reconstruction to reduce the impact of text replacement on the background. Specifically, the text-background separation module predicts the text mask of the scene text image, separating the source text from the background. Subsequently, the style-guided text transfer decoding module transfers the geometric and stylistic attributes of the source text to the content text, resulting in the target text. Next, the background and target text are combined to determine the minimal reconstruction area. Finally, the context-focused background reconstruction module is applied to the reconstruction area, producing the editing result. Furthermore, to ensure stable joint optimization of the four modules, a task-adaptive training optimization strategy has been devised. Experimental evaluations conducted on two popular datasets demonstrate the effectiveness of our approach. STEEM outperforms state-of-the-art methods, as evidenced by a reduction in the FID index from 29.48 to 24.67 and an increase in text recognition accuracy from 76.8% to 78.8%.

中文翻译:


显式解耦的文本传输,最大限度地减少了场景文本编辑的背景重建



场景文本编辑旨在将源文本替换为目标文本,同时保留原始背景。它的实际应用跨越了数据生成和隐私保护等各个领域,凸显了其近年来日益增长的重要性。在这项研究中,我们提出了一种新颖的场景文本编辑网络,具有显式解耦的文本传输和最小化的背景重建,称为 STEEM。与通常融合文本样式、文本内容和背景的现有方法不同,我们的方法侧重于将文本样式和内容与背景分离,并利用最小化的背景重建来减少文本替换对背景的影响。具体来说,text-background separation 模块可以预测场景文本图片的文本掩码,将源文本与背景分离。随后,样式引导式文本传输解码模块将源文本的几何和样式属性传输到内容文本,从而生成目标文本。接下来,将背景文本和目标文本组合在一起,以确定最小重建区域。最后,将基于上下文的背景重建模块应用于重建区域,产生编辑结果。此外,为保证 4 个模块的稳定联合优化,设计了一种任务自适应训练优化策略。在两个流行的数据集上进行的实验评估证明了我们方法的有效性。STEEM 的性能优于最先进的方法,FID 指数从 29.48 降低到 24.67,文本识别准确率从 76.8% 提高到 78.8%。
更新日期:2024-10-15
down
wechat
bug