EMO

EMO (Emote Portrait Alive), from Alibaba's Institute for Intelligent Computing (February 2024), was the paper that reset expectations for talking-head video. Its direct audio-to-video diffusion approach skipped the 3D face models and facial landmarks every prior method depended on: a ReferenceNet encodes identity from one portrait, an audio encoder reads the vocal track, and the backbone generates expressive facial motion and head poses frame by frame. Trained on 250+ hours of talking and singing footage across English, Mandarin, Japanese, Cantonese, and Korean, it produced singing performances, not just talking, with a naturalness that went viral and spawned a wave of follow-up work.

EMO is a research artifact, not a product: Alibaba never released official weights, code, or a hosted API, so everything in the wild is a community reimplementation or a third-party demo. That makes it historically important but practically frozen; for working open alternatives, EchoMimic (also Alibaba/Ant lineage) and Hallo3 are the maintained descendants. No benchmark scores were published, only author-reported comparisons against prior methods.

No published benchmark scores; superiority claims over prior methods are author-reported