Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrases

Published in ECCV, 2024

Motion understanding aims to map motion to action semantics, but the variability in both makes this challenging. Abstract actions like 'walk forwards' can be conveyed by diverse motions, while a single motion can have different meanings depending on context. Previous direct-mapping methods are unreliable, and current metrics fail to consistently assess motion-semantics alignment. To bridge this gap, we propose Kinematic Phrases (KP), which abstractly and objectively represent human motion with interpretability and generality. Using KP, we unify a motion knowledge base and build a motion understanding system. KP also enables Kinematic Prompt Generation (KPG), a novel benchmark for automatic motion generation. Experiments show our approach outperforms others, and we plan to release our code and data publicly.

Recommended citation: Liu, X., Li, Y. L., Zeng, A., Zhou, Z., You, Y., & Lu, C. (2023). Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrases. arXiv preprint arXiv:2310.04189.