Falcon 7B

Falcon 7B is the 7-billion-parameter causal decoder-only model from Abu Dhabi's Technology Innovation Institute, released in May 2023. Trained on 1,500 billion tokens of RefinedWeb enhanced with curated corpora, it topped comparable open models of its era on the OpenLLM Leaderboard while shipping under the permissive Apache 2.0 license with no royalties or restrictions.\n\nIts inference-optimized architecture uses FlashAttention and multi-query attention, and it became a standard fine-tuning base for the open-source community, spawning Falcon-7B-Instruct and a large ecosystem of derivatives.\n\nAs a raw pretrained model it needs fine-tuning for chat-style use, but its commercial-friendly license and strong-for-its-time performance made it one of 2023's defining open LLMs.