VeRL
Production-ready reinforcement learning training library for large language models.
About
VeRL, short for Volcano Engine Reinforcement Learning, is an open-source RL training library initiated by ByteDance's Seed team and maintained by the verl community. It is the open-source implementation of the HybridFlow framework presented at EuroSys 2025 and has been used to train models up to 671B parameters.
It supports algorithms including PPO, GRPO, DAPO, RLOO, and REINFORCE++, with FSDP and Megatron training backends and vLLM or SGLang rollout engines. Multi-turn rollouts with tool calling, vision-language model RL, and LoRA RL make it a strong fit for post-training agentic models.
What it can do
- PPO, GRPO, DAPO, RLOO, and more RL algorithms
- FSDP and Megatron backends with vLLM/SGLang rollouts
- Multi-turn tool-calling rollouts
- Scales to hundreds of GPUs and 671B models
Setup
Install with pip install verl[vllm] or pull the Docker image, then follow the GRPO and PPO trainer examples in the repository.
Prerequisites
Python 3.10+, PyTorch, Ray; GPUs for training; a base model checkpoint.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 116; Trending score 80/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.