Liger Kernel
Efficient Triton kernels for LLM training: higher throughput, far less memory.
About
Liger Kernel is a collection of Triton kernels designed specifically for LLM training, built by LinkedIn's GPU efficient runtime team. Fused implementations of RMSNorm, RoPE, SwiGLU, cross-entropy, and post-training losses like DPO and ORPO deliver roughly 20% higher multi-GPU training throughput and about 60% lower memory usage compared to standard Hugging Face implementations.
The kernels are Hugging Face compatible and work out of the box with FlashAttention, PyTorch FSDP, and DeepSpeed, with one-line integration in TRL trainers (use_liger_kernel=True). Kernels are correctness-tested against reference implementations, but as with any fused-kernel library, users should validate convergence on their own workloads before large runs.
What it can do
- Fused Triton kernels for LLM ops
- ~20% training throughput increase
- ~60% GPU memory reduction
- One-line TRL integration
- Post-training loss kernels (DPO, ORPO, SimPO)
Setup
pip install liger-kernel, then set use_liger_kernel=True in TRL trainer configs.
Prerequisites
A Hugging Face transformers or TRL training setup with Triton-compatible GPUs.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 79; Trending score 82/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.