PEFT
Parameter-efficient fine-tuning: LoRA and friends for every HF model.
About
PEFT is Hugging Face's library for parameter-efficient fine-tuning: adapt giant pretrained models by training a tiny fraction of their parameters — LoRA/QLoRA adapters, IA³, prompt tuning, prefix tuning — with performance close to full fine-tuning at a fraction of the compute and storage. One get_peft_model API wraps any Transformers model (0.19% of parameters trainable in the canonical example), and adapters plug into Diffusers and Accelerate for image models and distributed training. If you fine-tune open models, you almost certainly use this under the hood. Apache-2.0, 8.3M PyPI downloads/month.
What it can do
- **LoRA/QLoRA**: low-rank adapters — the default efficient fine-tune
- **Many methods**: IA³, AdaLoRA, prompt/prefix tuning, LoHa, LoKr, OFT, and more
- **Tiny checkpoints**: adapters are MBs, shareable and composable
- **Transformers-native**: works with Trainer, TRL, and Accelerate
- **Diffusers support**: manage adapters for image models
- **Merge or serve**: merge adapters into base weights or serve them hot-swapped
Setup
```bash
pip install peft
```
```python
from transformers import AutoModelForCausalLM
from peft import LoraConfig, TaskType, get_peft_model
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = get_peft_model(model, LoraConfig(task_type=TaskType.CAUSAL_LM, r=16))
# train model as usual — only the adapter weights update
```
Docs: huggingface.co/docs/peft. Adapters merge back or export separately.
Prerequisites
Python 3.9+; PyTorch; a base model from the Hub. GPU recommended (adapters train on modest VRAM).
Use cases
Works with
Trust score — 9/10
Scored 9/10 (verified tier, metrics-v1). Evidence: 21,780 GitHub stars (log-scaled vs catalog peers); blended with ~8,262,388 monthly package downloads; last push 0d ago; 8 releases/12mo; 117 open issues. Manual review: HF's PEFT library (LoRA/QLoRA/IA3); 21.8k stars, 8.3M PyPI/mo; it's the adapter layer, not a full training framework. License: Apache-2.0.
★ 21,780 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.7/10
- Maintenance · recency, releases, issue load
- 9.2/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 9.0/10
- · 21,780 GitHub stars (log-scaled vs catalog peers)
- · blended with ~8,262,388 monthly package downloads
- · last push 0d ago; 8 releases/12mo; 117 open issues
- · manual review: HF's PEFT library (LoRA/QLoRA/IA3); 21.8k stars, 8.3M PyPI/mo; it's the adapter layer, not a full training framework
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.