VALL-E X
Microsoft's cross-lingual codec LM: speak any of English, Chinese or Japanese in the prompt speaker's own voice.
About
VALL-E X is Microsoft's 2023 extension of VALL-E into cross-lingual speech synthesis. It trains a multilingual neural codec language model that takes a speech prompt in one language plus text in another and synthesizes the target language in the same voice — zero-shot, from a single utterance, preserving the speaker's timbre, emotion and even the acoustic environment. A language-ID token lets it control the foreign accent. The paper (arXiv 2303.03926, March 2023) reports wins over strong baselines on speaker similarity, word error rate, translation quality and naturalness.
Like the original VALL-E, Microsoft never released the model weights — only the paper and demos — so this is a research entry, not something you can call or download officially. A community reimplementation appeared on GitHub in August 2023, but it is unofficial. All performance claims are author-reported.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Author-reported: beats baselines on speaker similarity, WER, BLEU and naturalness; no independent verification
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (TIER verified, metrics-v1). Evidence: Microsoft Research paper (arXiv 2303.03926, March 2023) with baseline comparisons; official demo page (aka.ms/vallex); Microsoft Research publication record. Strengths: influential cross-lingual zero-shot approach, strong reported results, accent control. Docked for: (1) Microsoft released no weights or API — not usable as a product; (2) all claims author-reported; (3) research-only, no maintenance. License: no release (paper all rights reserved, Microsoft).
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.0/10
- Maintenance · recency, releases, issue load
- 4.0/10
- Openness · license + docs
- 4.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Paper arXiv 2303.03926, March 2023; cross-lingual neural codec language model (EN/ZH/JA)
- · Zero-shot cross-lingual TTS + speech-to-speech translation; language-ID accent control
- · Microsoft released no weights or API; only unofficial community reimplementation exists
- · No logo asset; logoUrl omitted