AI & ML
emg2face: Expressive Facial Animation with High-Density Surface EMG
arXiv cs.GR0 views
arXiv:2610.09304v1 Announce Type: new Abstract: Facial movements convey subtle and important information that is critical for human social communication. Optical methods for face capture are difficult or impossible to use when the face is occluded by head-mounted devices (HMDs), such as VR headsets. Even with a clear line of sight, such methods raise privacy concerns and require head-mounted capture rigs that offset cameras and lighting from the face. We show that high-density surface electromyography (HD-sEMG) provides a viable non-optical alternative that addresses these challenges. We measured 64 EMG channels, using two textile EMG grids, with 32 from the forehead (typically occluded by an HMD) and 32 from the side of the face. EMG data were digitized at 2048 Hz and filtered. Facial movements were simultaneously recorded and used to estimate 478 3D facial landmarks using MediaPipe's Face Landmarker. A major challenge in such multimodal recordings is synchronizing EMG and video data, which have different sampling frequencies and independent clocks. We developed a novel synchronization method using analog audio bursts that is capable of sub-millisecond synchronization. We also developed a staged fitting method that fits a recent high-resolution parametric head model (GNM), with 253 identity blendshapes and 383 expression blendshapes, to the MediaPipe landmarks as participants performed different facial expressions. We trained a deep neural network comprising per-grid spatial encoders followed by a dilated temporal convolutional network (TCN) to predict blendshape parameters from HD-sEMG signals at 100 Hz. Once trained, the network can predict expression blendshapes solely from HD-sEMG recordings. The output can be rendered using standard real-time blendshape animation methods. We demonstrate the methods using recordings from 25 participants, and direct expression transfer to a variety of human faces and non-human characters.
Source:https://arxiv.org/abs/2610.09304PreviousPhysics-based Sphere Packing for Lagrangian Mesh MorphingNextScribbleEdit: A Benchmark for Scribble-Only Image Editing
Related posts
OpenAI is finally merging ChatGPT’s chat and Work tabs
0TechCrunch

10 Open-Source Robotics Repositories for 3D Perception, VLA Policies, and Sensor Calibration
0dev.to
My self-modifying system tried to wire a gate-passing module into its own protection enforcer. It was refused three times.
0dev.to
Last night an AI agent performed brain surgery on a mouse
0Hacker News
The Card-Update Page Gets One Visit. Design for That Visit.
0dev.to UX
SaaS Partner Scout: A Script for Identifying GitHub SaaS Project Contributors
0dev.to GitHub