T5

T5 (Text-to-Text Transfer Transformer) is Google's 2019 framework that reframed every NLP task — translation, summarization, question answering, classification — as text-in, text-out. Pretrained on the C4 web corpus and released October 23, 2019, it came in sizes from 60M to 11B parameters and set new state-of-the-art results across the GLUE, SuperGLUE, and SQuAD benchmarks.

Its encoder-decoder architecture and unified framing made it enormously influential: the idea that one model with one interface could handle all language tasks directly inspired later instruction-tuned models, including the original Imagen's text encoder. T5 and its variants (T5-large, T5-3B, T5-11B, Flan-T5) remain workhorses for structured text tasks.

Everything is genuinely open: the paper, the C4 dataset recipe, the code, and the weights are all Apache 2.0, and the models run easily on modern hardware.