Phind-CodeLlama-34B-v2
Phind's instruction-tuned CodeLlama 34B that briefly held the open-source HumanEval crown at 73.8%.
About
Phind-CodeLlama-34B-v2 is a fine-tune of Meta's CodeLlama-34B built by the Phind team, the startup behind the AI search engine for developers. They trained it on a proprietary 1.5B-token dataset of programming problems and solutions structured as instruction-answer pairs, in 15 hours on 32 A100-80GB GPUs, and instruction-tuned it on the Alpaca/Vicuna format so it is steerable out of the box.
At launch it scored 73.8% pass@1 on HumanEval, which Phind claimed was state of the art among open-source models and ahead of GPT-4's 67%. The weights are gated behind Meta's Llama 2 Community License, which limits commercial use, and Phind never released the training data. It remains a popular self-hosted coding model on Ollama and GGUF quants, even though newer models have overtaken its scores.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- HumanEval: 73.8% pass@1, claimed SOTA among open models at release (provider-reported, HF model card)
- v1 reached 67.6-69.5% vs GPT-4's 67% on the same bench (provider-reported)
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: official Phind model card documents training (1.5B tokens, no LoRA, decontamination claimed) and the 73.8% HumanEval figure; license terms confirmed on Ollama's registry copy. Strengths: historic SOTA result, wide community adoption, easy local deployment. Docked for: (1) training data proprietary and unreleased, so the score cannot be independently replicated; (2) Llama 2 Community License restricts commercial use; (3) superseded by newer open models.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 5.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Historic SOTA open-model HumanEval (73.8%) at launch
- · Training data proprietary and unreleased; superseded by newer models