Construction sites are dynamic with various activities and equipment sounds, essential for identifying equipment, understanding work processes, and assessing site conditions. However, recognizing equipment actions using audio data faces challenges like manual recording dependency, collecting high-quality datasets, and background noise. This paper introduces an automated framework, aided by computer vision algorithms, for generating an audio dataset for construction equipment from online sources. The framework uses computer vision to identify relevant visual content and audio classification models to filter out irrelevant content, ensuring high-quality data. Through the proposed framework, an audio dataset was generated with annotations covering equipment types and actions. Performance evaluation with classification models showed F-scores ranging from 61 % to 91 % at the equipment level and 52 % to 87 % at the action level. The framework offers an effective approach to creating audio datasets, supporting advancements in audio-based activity recognition, contributing to improvements in real-world construction site safety and productivity.

中文翻译：

---

计算机视觉辅助音频数据集生成，用于识别建筑设备动作


建筑工地是动态的，有各种活动和设备声音，对于识别设备、了解工作流程和评估工地条件至关重要。但是，使用音频数据识别设备作面临手动录制依赖性、收集高质量数据集和背景噪音等挑战。本文介绍了一个由计算机视觉算法辅助的自动化框架，用于从在线资源为建筑设备生成音频数据集。该框架使用计算机视觉来识别相关的视觉内容，并使用音频分类模型来过滤掉不相关的内容，从而确保高质量的数据。通过提出的框架，生成了一个音频数据集，其中包含涵盖设备类型和动作的注释。使用分类模型进行的性能评估显示，设备级别的 F 分数在 61% 到 91% 之间，在行动级别的 F 分数在 52% 到 87% 之间。该框架提供了一种创建音频数据集的有效方法，支持基于音频的活动识别的进步，有助于提高实际建筑工地的安全性和生产力。