This paper introduces ELMO, a real-time upsampling motion capture framework designed for a single LiDAR sensor. Modeled as a conditional autoregressive transformer-based upsampling motion generator, ELMO achieves 60 fps motion capture from a 20 fps LiDAR point cloud sequence. The key feature of ELMO is the coupling of the self-attention mechanism with thoughtfully designed embedding modules for motion and point clouds, significantly elevating the motion quality. To facilitate accurate motion capture, we develop a one-time skeleton calibration model capable of predicting user skeleton off-sets from a single-frame point cloud. Additionally, we introduce a novel data augmentation technique utilizing a LiDAR simulator, which enhances global root tracking to improve environmental understanding. To demonstrate the effectiveness of our method, we compare ELMO with state-of-the-art methods in both image-based and point cloud-based motion capture. We further conduct an ablation study to validate our design principles. ELMO's fast inference time makes it well-suited for real-time applications, exemplified in our demo video featuring live streaming and interactive gaming scenarios. Furthermore, we contribute a high-quality LiDAR-mocap synchronized dataset comprising 20 different subjects performing a range of motions, which can serve as a valuable resource for future research. The dataset and evaluation code are available at https://movin3d.github.io/ELMO_SIGASIA2024/

中文翻译：

---

ELMO：通过上采样增强实时 LiDAR 动作捕捉


本文介绍了 ELMO，这是一个专为单个 LiDAR 传感器设计的实时上采样运动捕捉框架。ELMO 建模为基于条件自回归变压器的上采样运动生成器，可从 20 fps 的 LiDAR 点云序列中实现 60 fps 的运动捕捉。ELMO 的主要特点是将自我注意机制与精心设计的运动和点云嵌入模块相结合，显著提高了运动质量。为了促进准确的动作捕捉，我们开发了一个一次性骨骼校准模型，能够从单帧点云中预测用户骨骼偏移。此外，我们还引入了一种利用 LiDAR 模拟器的新型数据增强技术，该技术增强了全局根跟踪以提高对环境的理解。为了证明我们方法的有效性，我们将 ELMO 与基于图像和基于点云的运动捕捉中的最先进的方法进行了比较。我们进一步进行了消融研究以验证我们的设计原则。ELMO 的快速推理时间使其非常适合实时应用程序，我们的演示视频中展示了实时流媒体和交互式游戏场景。此外，我们还提供了一个高质量的 LiDAR 动作捕捉同步数据集，包括 20 个不同的受试者执行一系列运动，可以作为未来研究的宝贵资源。数据集和评估代码可在 https://movin3d.github.io/ELMO_SIGASIA2024/