Unsupervised learning of style-aware facial animation from real acting performances

Unsupervised learning of style-aware facial animation from real acting performances

Blog Article

This paper presents a invertatop squeeze bottle novel approach for text/speech-driven animation of a photo-realistic head model based on blend-shape geometry, dynamic textures, and neural rendering.Training a VAE for geometry and texture yields a parametric model for accurate capturing and realistic synthesis of facial expressions from a latent feature vector.Our animation method is based on a conditional CNN that transforms text or speech into a sequence of animation parameters.In contrast to previous approaches, our animation model learns disentangling/synthesizing different acting-styles in an unsupervised manner, requiring only phonetic labels here that describe the content of training sequences.

For realistic real-time rendering, we train a U-Net that refines rasterization-based renderings by computing improved pixel colors and a foreground matte.We compare our framework qualitatively/quantitatively against recent methods for head modeling as well as facial animation and evaluate the perceived rendering/animation quality in a user-study, which indicates large improvements compared to state-of-the-art approaches.