TinyLlama 1.1B

TinyLlama 1.1B is an open-source 1.1-billion-parameter language model pretrained from scratch on roughly 3 trillion tokens of SlimPajama and StarCoder data. It adopts exactly the Llama 2 architecture and tokenizer, so it plugs directly into the Llama tooling ecosystem, and the full training run was completed in about 90 days on 16 A100 GPUs.

Its value is accessibility rather than raw capability: the model runs comfortably on consumer hardware and even CPUs, it ships under the permissive Apache 2.0 license, and it has one of the largest community fine-tune and quantization ecosystems of any small model, including widely used GGUF and GPTQ builds. Later releases added a chat-tuned variant trained with the Zephyr recipe and DPO alignment on UltraFeedback.