We present EgoHDM, an online egocentric-inertial human motion capture (mocap), localization, and dense mapping system. Our system uses 6 inertial measurement units (IMUs) and a commodity head-mounted RGB camera. EgoHDM is the first human mocap system that offers dense scene mapping in near real-time. Further, it is fast and robust to initialize and fully closes the loop between physically plausible map-aware global human motion estimation and mocap-aware 3D scene reconstruction. To achieve this, we design a tightly coupled mocap-aware dense bundle adjustment and physics-based body pose correction module leveraging a local body-centric elevation map. The latter introduces a novel terrain-aware contact PD controller, which enables characters to physically contact the given local elevation map thereby reducing human floating or penetration. We demonstrate the performance of our system on established synthetic and real-world benchmarks. The results show that our method reduces human localization, camera pose, and mapping accuracy error by 41%, 71%, 46%, respectively, compared to the state of the art. Our qualitative evaluations on newly captured data further demonstrate that EgoHDM can cover challenging scenarios in non-flat terrain including stepping over stairs and outdoor scenes in the wild. Our project page: https://handiyin.github.io/EgoHDM/

中文翻译：

---

EgoHDM：实时以自我为中心的惯性人体动作捕捉、定位和密集映射系统


我们介绍了 EgoHDM，这是一种在线以自我为中心的惯性人体动作捕捉 （mocap）、定位和密集映射系统。我们的系统使用 6 个惯性测量单元 （IMU） 和一个商用头戴式 RGB 相机。EgoHDM 是第一个近乎实时地提供密集场景映射的人类动作捕捉系统。此外，它初始化快速而健壮，并在物理上合理的地图感知全局人体运动估计和 mocap 感知 3D 场景重建之间完全闭合循环。为了实现这一目标，我们设计了一个紧密耦合的 mocap 感知密集束调整和基于物理的身体姿势校正模块，利用以身体为中心的局部海拔图。后者引入了一种新颖的地形感知接触 PD 控制器，使角色能够物理接触给定的局部高程图，从而减少人类漂浮或穿透。我们根据既定的综合和真实基准来展示我们的系统性能。结果表明，与最先进的方法相比，我们的方法将人类定位、摄像机姿态和地图绘制精度误差分别减少了 41%、71%、46%。我们对新捕获的数据的定性评估进一步表明，EgoHDM 可以覆盖非平坦地形中具有挑战性的场景，包括跨过楼梯和野外的户外场景。我们的项目页面： https://handiyin.github.io/EgoHDM/