Existing augmented reality (AR) assembly mainly provides visual instructions for operators from a first-person perspective, and it is hard to share individual working intents for co-located workers on the shop floor, especially for large-scale product assembly task that requires multiple operators working together. To bridge this gap for practical deployments, this paper proposes CoiAR, a co-located audio-visual enabled mobile collaborative AR assembly. Firstly, according to the stereo visual-inertial fusion strategy, robust and accurate self-contained motion tracking is achieved for the resource-constrained mobile AR platform, followed by a co-located alignment from multiple mobile AR clients on the shop floor. Then, a lightweight text-aware network for online wiring harness character recognition is proposed, as well as the audio-based confirming strategy, enabling natural audio-visual interaction among co-located workers within a shared immersive workplace, which can also monitor the current wiring assembly status and activate the step-by-step tutorials automatically. The novelty of this work is focused on the deployment of audio-visual aware interaction using the same device that is being used to deploy the co-located collaborative AR work instructions, establishing shared operating intents among multiple co-located workers. Finally, comprehensive experiments are carried out on the collaborative performance among multiple AR clients, and results illustrate that the proposed CoiAR can alleviate the cognitive load and achieve superior performance for the co-located AR assembly tasks, providing a more human-centric collaborative assembly performance.

中文翻译：

---

Co2iAR：协同定位视听支持的移动协作工业 AR 线束组件


现有的增强现​​实（AR）装配主要以第一人称视角为操作人员提供视觉指导，车间内共处一地的工人很难分享个人的工作意图，特别是对于需要多个人的大规模产品装配任务运营商一起工作。为了弥补实际部署中的这一差距，本文提出了 CoiAR，一种协同定位的视听移动协作 AR 组件。首先，根据立体视觉惯性融合策略，为资源有限的移动AR平台实现鲁棒且准确的独立运动跟踪，然后在车间内实现多个移动AR客户端的共定位对齐。然后，提出了一种用于在线线束字符识别的轻量级文本感知网络，以及基于音频的确认策略，使同地办公的工作人员在共享的沉浸式工作场所内能够进行自然的视听交互，还可以监控当前的状态接线组件状态并自动激活分步教程。这项工作的新颖之处在于使用与部署同地协作 AR 工作指令相同的设备来部署视听感知交互，从而在多个同地工作人员之间建立共享的操作意图。最后，对多个 AR 客户端之间的协作性能进行了全面的实验，结果表明，所提出的 CoiAR 可以减轻认知负荷，并为共定位的 AR 装配任务实现卓越的性能，提供更加以人为中心的协作装配性能。