当前位置: X-MOL 学术Autom. Constr. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Weakly-aligned cross-modal learning framework for subsurface defect segmentation on building façades using UAVs
Automation in Construction ( IF 9.6 ) Pub Date : 2025-01-08 , DOI: 10.1016/j.autcon.2024.105946
Sudao He, Gang Zhao, Jun Chen, Shenghan Zhang, Dhanada Mishra, Matthew Ming-Fai Yuen

Infrared (IR) thermography combined with Unmanned Aerial Vehicles (UAVs) offers an innovative approach for automated building façades inspections. However, extracting quantitative defect information from a single image poses a significant challenge. To address this, this paper introduces a Weakly-aligned Cross-modal Learning framework for subsurface defect segmentation using UAVs. This framework consists of two main components: the Multimodal Feature Description Network (MFDN) and the Prompt-aided Cross-modal Graph Learning (PCGL) algorithm. Initially, RGB–IR image pairs are processed by MFDN to extract feature descriptors for multi-modal alignment. The PCGL algorithm identifies visually critical areas through graph partitioning on a Wasserstein graph. These critical areas are transferred to the aligned IR image, and a Wasserstein Adjacency Graph (WAG) is constructed based on masked superpixel segmentation. Finally, the defects contours are pinpointed by detecting abnormal vertices of the WAG. The effectiveness is validated through controlled laboratory experiments and field applications on tiled façades.

中文翻译:


使用无人机对建筑物立面进行地下缺陷分割的弱对齐跨模态学习框架



红外 (IR) 热成像技术与无人机 (UAV) 相结合,为建筑外墙的自动化检测提供了一种创新方法。然而,从单个图像中提取定量缺陷信息是一项重大挑战。为了解决这个问题,本文介绍了一个使用无人机进行地下缺陷分割的弱对齐跨模态学习框架。该框架由两个主要组件组成:多模态特征描述网络 (MFDN) 和提示辅助跨模态图学习 (PCGL) 算法。最初,RGB-IR 图像对由 MFDN 处理,以提取用于多模态对齐的特征描述符。PCGL 算法通过在 Wasserstein 图上进行图形分区来识别视觉上的关键区域。这些关键区域被传输到对齐的红外图像,并基于掩蔽超像素分割构建 Wasserstein 邻接图 (WAG)。最后,通过检测 WAG 的异常顶点来精确定位缺陷轮廓。通过受控的实验室实验和瓷砖立面的现场应用验证了其有效性。
更新日期:2025-01-08
down
wechat
bug