torchtune
PyTorch's native library for fine-tuning LLMs.
About
torchtune is PyTorch's native fine-tuning library: SFT, RLHF, DPO, and LoRA/QLoRA recipes built on torch's latest (FSDP2, torch.compile, efficient attention) — the 'PyTorch way' to fine-tune Llama, Qwen, Gemma, and friends. 5.8k stars, BSD-3-Clause, now under the meta-pytorch org (moved from pytorch; old URLs redirect). It's code-first — no LlamaFactory-style web UI — and the community is smaller than trl/axolotl's, but for PyTorch-native teams it's the cleanest stack.
What it can do
- **Native PyTorch**: FSDP2, torch.compile, latest torch features
- **All methods**: full FT, LoRA/QLoRA, DPO, PPO, RLOO
- **Recipes**: YAML configs for common models
- **Multimodal**: vision-language fine-tuning
- **Composable**: hackable, library-first design
Setup
```bash
pip install torchtune
tune download meta-llama/Meta-Llama-3.1-8B-Instruct --output-dir /tmp/llama3_1_8B --hf-token <TOKEN>
tune run lora_finetune_single_device --config llama3_1/8B_lora_single_device
```
Docs: pytorch.org/torchtune.
Prerequisites
Python 3.10+; PyTorch 2.4+; NVIDIA GPU. HF token for gated models.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: 5,812 GitHub stars (log-scaled vs catalog peers); last push 33d ago; 0 releases/12mo; 465 open issues. Manual review: PyTorch's native fine-tuning (SFT/RLHF/DPO/LoRA, FSDP2), 5.8k stars, moved to meta-pytorch org; code-first, smaller community than trl/axolotl. License: BSD-3-Clause.
★ 5,812 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.2/10
- Maintenance · recency, releases, issue load
- 5.1/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · 5,812 GitHub stars (log-scaled vs catalog peers)
- · last push 33d ago; 0 releases/12mo; 465 open issues
- · manual review: PyTorch's native fine-tuning (SFT/RLHF/DPO/LoRA, FSDP2), 5.8k stars, moved to meta-pytorch org; code-first, smaller community than trl/axolotl
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.