Open Source
TalkFlow: Structured Latent Flow Matching for Speech Driven Holistic Motion Generation
arXiv cs.GR0 views
arXiv:2610.40972v1 Announce Type: new Abstract: Holistic co-speech motion generation aims to synthesize coordinated facial expressions, hand gestures, and body movements from speech, which is challenging due to heterogeneous part-specific dynamics, long-range temporal coherence, and cross-part coordination. Directly modeling high-dimensional motion can be unstable, while overly compact or unstructured latent spaces may lose fine-grained motion details. We present \textbf{TalkFlow}, a structured latent flow framework for speech-driven holistic motion generation. TalkFlow learns a continuous three-stream latent representation for upper-body motion, lower-body motion, and facial expressions, preserving part-specific dynamics without forcing heterogeneous motion components into a single latent space. To maintain holistic coordination, lightweight cross-stream interactions are incorporated in both latent representation learning and flow-based generation. Conditioned on multi-level speech features, TalkFlow performs speech-conditioned latent flow matching, learning a continuous transport process from source latents to realistic motion latents. Experiments on the BEAT2 dataset demonstrate strong performance across facial expression accuracy, body gesture quality, and speech-motion synchronization. Qualitative comparisons and user studies further show that TalkFlow generates more natural, expressive, and speech-aligned holistic motion than state-of-the-art methods. The project page is available at https://xinmuwang.github.io/TalkFlow/.
Source:https://arxiv.org/abs/2610.40972PreviousBest 7 Real-Time AI Face Swap Tools in 2026: A Buyer's GuideNextDynamic Skinning: Kinematics-Driven Cartoon Effects for Articulated Characters
Related posts
Astra can read and write weights to improve Qwen
0Hacker News
Distillation didn't help our small model until we took the labels away
0dev.to Machine Learning

462. Minimum Moves to Equal Array Elements II
0dev.to PHP
MUSBRIC: LEGO-Compatible Pneumatic Muscle Bricks for Reconfigurable Musculoskeletal Robots
0arXiv cs.RO
OpenFlyV2: An onboard vision-language-action model for aerial perception and navigation
0arXiv cs.RO
Show HN: Open Bot Avatars – a spec and renderer for interactive bot avatars
0Hacker News New