LiDAR-based human motion capture has garnered significant interest in recent years for its practicability in large-scale and unconstrained environments. However, most methods rely on cleanly segmented human point clouds as input, the accuracy and smoothness of their motion results are compromised when faced with noisy data, rendering them unsuitable for practical applications. To address these limitations and enhance the robustness and precision of motion capture with noise interference, we introduce LiveHPS++, an innovative and effective solution based on a single LiDAR system. Benefiting from three meticulously designed modules, our method can learn dynamic and kinematic features from human movements, and further enable the precise capture of coherent human motions in open settings, making it highly applicable to real-world scenarios. Through extensive experiments, LiveHPS++ has proven to significantly surpass existing state-of-the-art methods across various datasets, establishing a new benchmark in the field.
The pipeline of LiveHPS++. It consists of three primary modules, including a trajectory-guided body tracker to predict the human joint and translation, a noise-insensitive velocity predictor to regress the velocity, and the kinematic-aware pose optimizer to enhance the accuracy and coherence of results. Finally, we use SMPL solver to regress the parameters of human poses and shape. Detailed network structure of three modules is also shown under the upper pipeline.
The NoiseMotion dataset simulation pipeline, integrating dynamic human motion and static object noise to simulate real-world human-object interactions.
NoiseMotion |── surreal | |── pc_1 (View-1) | | |── train | | | |── run1 | | | | |── 01_01.pkl | | | | |── 01_02.pkl | | | | |── ... | | |—— test | | | |── run1 | | | | |── 03_01.pkl | | | | |── 03_02.pkl | | | | |── ... | |── pc_2 (View-2) | |── pc_3 (View-3)
Figure 1. Visualization of the motion capture performance of LiveHPS++ in a real-time captured scenario with complex human-object interaction. The left exhibits images for reference, the middle shows the noised point clouds (top) and our corresponding mesh model results (bottom). We zoom in some cases on the right for clearer demonstration, where point clouds are drawn in white.
np.load(file_path, allow_pickle=True) to load the file.
@article{ren2024livehps++,
title={LiveHPS++: Robust and Coherent Motion Capture in Dynamic Free Environment},
author={Ren, Yiming and Han, Xiao and Yao, Yichen and Long, Xiaoxiao and Sun, Yujing and Ma, Yuexin},
journal={arXiv preprint arXiv:2407.09833},
year={2024}
}
}