XCOMET
Explainable neural MT metric that scores translations and pinpoints error spans, with the strongest correlation to human MQM judgments.
About
XCOMET is Unbabel's explainable take on machine translation evaluation. Unlike a plain regression score, it predicts both a quality score and fine-grained error spans with severity labels, so developers can see exactly what a translation got wrong. It comes in XL (3.5B) and XXL (10.7B) sizes and works in reference-based and reference-free (quality estimation) modes, covering over a hundred languages.
The authors report the highest correlation with human MQM judgments among published metrics at WMT23, which made XCOMET the default choice for serious MT evaluation and system ranking. The weights are CC-BY-NC-4.0, so commercial use is restricted, and the XXL variant needs serious GPU memory. It integrates with the open COMET toolkit for scoring, MBR decoding, and reranking.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Author-reported (WMT23 metrics shared task): highest correlation with human MQM judgments among evaluated metrics
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: Unbabel release with WMT23 metrics-shared-task results confirmed via the COMET toolkit docs and Hugging Face cards; XL (3.5B) and XXL (10.7B) checkpoints published. Strengths: best-published MQM correlation, error-span explainability, reference-free mode, standard evaluation tooling. Docked for: (1) CC-BY-NC-4.0 license blocks commercial use; (2) XXL size demands heavy GPU resources. License: CC-BY-NC-4.0.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 7.0/10
- Openness · license + docs
- 7.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Unbabel release; XL (3.5B) and XXL (10.7B) checkpoints
- · Highest published correlation with human MQM judgments at WMT23
- · Error-span explainability; reference-based and reference-free modes