LLaMA-Factory
Unified open-source fine-tuning framework for 100+ LLMs, with SFT, RLHF, and LoRA behind a no-code web UI.
About
LLaMA-Factory is an open-source fine-tuning toolkit by developer hiyouga that has become one of the default ways to adapt open models. It supports 100+ LLMs with supervised fine-tuning (including multimodal), reward modeling, and reinforcement learning (PPO, DPO, ORPO, KTO), plus parameter-efficient methods like LoRA, QLoRA, GaLore, and DoRA. Its LLaMA Board web UI lets users pick a model, upload a dataset, and launch training with no coding.
The project was described in an arXiv paper (2403.13372) reporting over 13,000 stars and 1,600 forks at publication time, and hundreds of community models on the Hugging Face Hub are built on it. It is Apache 2.0 licensed and especially central in the Chinese AI developer ecosystem, with first-class support for Qwen, DeepSeek, Yi, Baichuan, and ChatGLM. It remains a research-grade toolkit rather than a managed service: teams still need their own GPU infrastructure and dataset pipelines.
What it can do
- Fine-tuning for 100+ LLM architectures via one framework
- SFT, RLHF (PPO/DPO/ORPO/KTO), and reward modeling
- LoRA, QLoRA, GaLore, DoRA, and full 16-bit tuning
- No-code LLaMA Board web UI for zero-code workflows
- vLLM and DeepSpeed integration for scale
Setup
Clone https://github.com/hiyouga/LLaMA-Factory, pip install the requirements, and launch the Gradio-based LLaMA Board web UI or use the CLI/YAML configs.
Prerequisites
Python 3.8+; GPU infrastructure (CUDA); datasets in Alpaca, ShareGPT, or DPO format.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 75; Trending score 80/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.