This is a simplified guide to an AI model called Video-Retalking maintained by Chenxwh. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter.
Model overview
The video-retalking model, created by maintainer chenxwh, is an AI system that can edit the faces of a real-world talking head video according to input audio, producing a high-quality and lip-synced output video even with a different emotion. This model builds upon previous work like VideoReTalking, Wav2Lip, and GANimation, disentangling the task into three sequential steps: face video generation with a canonical expression, audio-driven lip-sync, and face enhancement for improving photorealism.
Model inputs and outputs
The video-retalking model takes two inputs: a talking-head video file and an audio file. It then outputs a new video file where the face in the original video is lip-synced to the input audio.
Inputs
- Face: Input video file of a talking-head
- Input Audio: Input audio file to drive the lip-sync
Outputs
- Output Video: New video file with the face lip-synced to the input audio
Capabilities
The video-retalking model is capable...
Top comments (0)