V-Express
Tencent's portrait-video model that solved the weak-signal problem: audio, pose, and reference image balanced through progressive training and conditional dropout for clean lip-synced video.
About
V-Express (June 2024, arXiv 2406.02511) is Tencent AI Lab's take on portrait video generation, and its contribution is a training insight rather than scale: when you condition on text, audio, reference image, pose, and depth together, weak signals like audio get drowned out by strong ones like the reference face. V-Express fixes it with progressive training plus conditional dropout — gradually teaching the model to actually listen to the weak conditions — built on a Stable Diffusion backbone with reference-attention, audio-attention, and temporal-attention layers. On TalkingHead-1KH and AVSpeech it was evaluated against Wav2Lip and DiffusedHeads on FID, FVD, identity similarity, pose alignment, and SyncNet lip-sync scores; it led on video quality and control alignment while trailing Wav2Lip on pure lip-sync. Code and weights are published on GitHub (tencent-ailab/V-Express), so it's one of the few audio-driven portrait models you can genuinely run. For dubbing pipelines that need a re-animated face to match translated audio, it's a practical open option.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- SyncNet lip-sync, FID, FVD, identity similarity vs Wav2Lip and DiffusedHeads on TalkingHead-1KH/AVSpeech (author-reported)\n- Trails Wav2Lip on lip-sync, leads on video quality and pose alignment (author-reported)\n- No independent benchmark replication found
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (TIER verified, metrics-v1). Evidence: peer-reviewed-style paper (arXiv 2406.02511) with head-to-head numbers against Wav2Lip/DiffusedHeads; public code and weights on GitHub. Strengths: genuinely runnable, honest about trailing on lip-sync, novel weak-signal training method. Docked for: (1) all benchmarks author-reported; (2) research project with no maintained product or support; (3) weight license terms not clearly documented — verify before commercial use. License: research release; check the repo license.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 6.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 8.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · arXiv 2406.02511; conditional dropout + progressive training balances weak audio signals
- · Benchmarked vs Wav2Lip/DiffusedHeads on TalkingHead-1KH/AVSpeech (author-reported); trails on lip-sync, leads on video quality
- · Public code and weights on GitHub (tencent-ailab/V-Express)
- · Weight license terms not clearly documented — verify before commercial use