In the context of human-centric smart manufacturing, human-robot collaboration (HRC) systems leverage the strengths of both humans and machines to achieve more flexible and efficient manufacturing. In particular, estimating and monitoring human motion status determines when and how the robots cooperate. However, the presence of occlusion in industrial settings seriously affects the performance of human pose estimation (HPE). Using more sensors can alleviate the occlusion issue, but it may cause additional computational costs and lower workers' comfort. To address this issue, this work proposes a visual-inertial fusion-based method for HPE in HRC, aiming to achieve accurate and robust estimation while minimizing the influence on human motion. A part-specific cross-modal fusion mechanism is designed to integrate spatial information provided by a monocular camera and six Inertial Measurement Units (IMUs). A multi-scale temporal module is developed to model the motion dependence between frames at different granularities. Our approach achieves 34.9 mm Mean Per Joint Positional Error (MPJPE) on the TotalCapture dataset and 53.9 mm on the 3DPW dataset, outperforming state-of-the-art visual-inertial fusion-based methods. Tests on a synthetic-occlusion dataset further validate the occlusion robustness of our network. Quantitative and qualitative experiments on a real assembly case verified the superiority and potential of our approach in HRC. It is expected that this work can be a reference for human motion perception in occluded HRC scenarios.

中文翻译：

---

一种基于深度学习的视觉惯性融合方法，用于闭塞人机协作装配场景中的人体姿态估计


在以人为本的智能制造背景下，人机协作 （HRC） 系统利用人和机器的优势来实现更灵活、更高效的制造。特别是，估计和监控人体运动状态决定了机器人何时以及如何合作。但是，工业环境中的遮挡会严重影响人体姿态估计 （HPE） 的性能。使用更多的传感器可以缓解遮挡问题，但可能会导致额外的计算成本并降低工人的舒适度。为了解决这个问题，这项工作提出了一种基于 HRC 的 HPE 视觉惯性融合方法，旨在实现准确和稳健的估计，同时最大限度地减少对人体运动的影响。设计了一种特定于部件的跨模态融合机制，以集成由单目摄像头和六个惯性测量单元 （IMU） 提供的空间信息。开发了一个多尺度时间模块来模拟不同粒度的帧之间的运动依赖性。我们的方法在 TotalCapture 数据集上实现了 34.9 毫米的每关节平均位置误差 （MPJPE），在 3DPW 数据集上实现了 53.9 毫米，优于最先进的基于视觉惯性聚变的方法。对合成遮挡数据集的测试进一步验证了我们网络的遮挡稳健性。在真实装配案例上的定量和定性实验验证了我们在 HRC 中的方法的优越性和潜力。预计这项工作可以成为闭塞 HRC 场景中人类运动感知的参考。