TRL
Post-train foundation models: SFT, DPO, GRPO, and RLHF trainers.
About
TRL (Transformer Reinforcement Learning) is Hugging Face's library for post-training foundation models: supervised fine-tuning (SFT), preference optimization (DPO, KTO, ORPO), and reinforcement learning (GRPO, RLOO, PPO) behind clean *Trainer APIs built on Transformers. Recent versions score tokens with a fused LM head by default — up to 6.9x longer sequences on the same GPU — and it integrates with Accelerate (multi-node), PEFT (LoRA/QLoRA on modest hardware), and Unsloth (fast kernels), plus a no-code CLI. Apache-2.0, 19.5k stars, 2.5M PyPI downloads/month.
What it can do
- **SFT/DPO/KTO/GRPO/RLOO/PPO trainers**: the full post-training menu
- **Fused LM head**: up to 6.9x longer sequences, 52-82% less peak VRAM at 8k tokens
- **PEFT integration**: LoRA/QLoRA training on modest hardware
- **Distributed**: Accelerate-powered DDP/DeepSpeed multi-node scaling
- **Unsloth integration**: optimized kernels for faster training
- **CLI**: fine-tune without writing code
Setup
```bash
pip install trl
```
```python
from trl import SFTTrainer, SFTConfig
# SFTTrainer(model, args=SFTConfig(...), train_dataset=ds).train()
```
Or no-code: trl sft --model_name_or_path Qwen/Qwen2.5-7B. Docs: huggingface.co/docs/trl.
Prerequisites
Python 3.9+; PyTorch. GPU(s) sized to the method (QLoRA via PEFT for consumer GPUs).
Use cases
Works with
Trust score — 9/10
Scored 9/10 (verified tier, metrics-v1). Evidence: 19,498 GitHub stars (log-scaled vs catalog peers); blended with ~2,540,661 monthly package downloads; last push 0d ago; 35 releases/12mo; 253 open issues. Manual review: HF post-training (SFT/DPO/GRPO/RLOO); fused LM head (6.9x sequences); 19.5k stars, 2.5M PyPI/mo; post-training only by design. License: Apache-2.0.
★ 19,498 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.4/10
- Maintenance · recency, releases, issue load
- 9.5/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 9.0/10
- · 19,498 GitHub stars (log-scaled vs catalog peers)
- · blended with ~2,540,661 monthly package downloads
- · last push 0d ago; 35 releases/12mo; 253 open issues
- · manual review: HF post-training (SFT/DPO/GRPO/RLOO); fused LM head (6.9x sequences); 19.5k stars, 2.5M PyPI/mo; post-training only by design
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.