FLAN-UL2

FLAN-UL2 applies the FLAN instruction-tuning recipe to Google's UL2 20B base model, released February 2023. It combines UL2's mixture-of-denoisers pretraining with large-scale instruction fine-tuning, and on most natural language understanding benchmarks it outperformed FLAN-T5 by a significant margin. It became a popular open research model for instruction-following work in the pre-ChatGPT open-model wave.

Checkpoints are public under Apache 2.0 (google/flan-ul2 on Hugging Face). As a 20B encoder-decoder it needs substantial memory but remains a strong baseline for instruction-tuning research.