COMETKiwi
Reference-free quality estimation for machine translation: score any translation using only the source text.
About
COMETKiwi is Unbabel's reference-free MT evaluation model. It estimates translation quality from the source sentence alone, no human reference needed, which makes it usable in production pipelines where references never exist: filtering MT output, flagging segments for human review, or reranking candidates in minimum Bayes risk decoding. The wmt22-cometkiwi-da checkpoint builds on InfoXLM, with XL and XXL upgrades trained on WMT23 data.
It was a top quality-estimation system at the WMT22 QE shared task and remains the standard reference-free baseline in the COMET toolkit. The weights are CC-BY-NC-4.0, so like the rest of the Unbabel metric family they are research-only for commercial purposes. Scores range from 0 to 1, with 1 meaning a perfect translation.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Author-reported (WMT22 QE shared task): top reference-free quality estimation system; larger wmt23 variants released for improved correlation
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: Unbabel release with WMT22 QE shared-task standing confirmed via the COMET toolkit docs and Hugging Face cards; wmt22, wmt23 XL (3.5B) and XXL (10.7B) checkpoints published. Strengths: the standard reference-free QE baseline, production-friendly use cases, mature toolkit integration. Docked for: (1) CC-BY-NC-4.0 license blocks commercial use; (2) reference-free scores are inherently noisier than reference-based metrics. License: CC-BY-NC-4.0.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 7.0/10
- Openness · license + docs
- 7.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Unbabel release; wmt22-cometkiwi-da plus wmt23 XL (3.5B) and XXL (10.7B) variants
- · Top reference-free QE system at the WMT22 QE shared task
- · Part of the open COMET toolkit