Magicoder
Open 7B-class code models trained on OSS-Instruct, synthetic instruction data seeded from real open-source snippets.
About
Magicoder is a family of open code models from the University of Illinois, built around a training-data idea called OSS-Instruct. Instead of asking a big model to dream up coding exercises from nothing, the researchers feed it real seed snippets from open source and have it invent diverse, realistic problems around them. The flagship Magicoder-DS-6.7B is a DeepSeek-Coder base fine-tuned on just 90M tokens of this data, and the CodeLlama-based Magicoder-CL-7B came from the same recipe.
On the authors' own numbers, Magicoder-DS-6.7B hits 66.5% pass@1 on HumanEval+ and 75.4% on MBPP+, competitive with far larger models of its era. Everything is open: MIT-licensed code and data on GitHub plus the weights and the 75K-entry OSS-Instruct dataset on Hugging Face. The paper was accepted at ICML 2024, which gives the data-generation claims more weight than a typical one-off release.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- HumanEval+: Magicoder-DS 6.7B 66.5% pass@1 (provider-reported, paper Table 4)
- MBPP+: Magicoder-DS 6.7B 75.4% pass@1 (provider-reported)
- DS-1000: Magicoder-CL 7B 29.9% overall pass@1 (provider-reported)
- Paper claims 8x fewer fine-tuning tokens than DeepSeek-Coder-Instruct-6.7B for better scores (provider-run comparison)
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: ICML'24 peer-reviewed paper (arXiv:2312.02120) with published methodology including decontamination; MIT-licensed code, weights, and training data; claims cross-checked against the EvalPlus leaderboard lineage. Strengths: novel OSS-Instruct data recipe, fully reproducible artifacts, strong small-model results. Docked for: (1) headline benchmark figures are author-run with no independent replication; (2) a university lab artifact with no production API and limited maintenance.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.0/10
- Maintenance · recency, releases, issue load
- 6.0/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · ICML'24 peer-reviewed paper with decontamination methodology
- · Weights, data, and code all public; EvalPlus leaderboard lineage