RedPajama-INCITE-7B

RedPajama-INCITE-7B is a 7-billion-parameter language model developed by Together with open-source community partners, trained on the RedPajama-Data-1T dataset. The project set out to replicate the LLaMA training recipe while keeping everything fully open under the Apache 2.0 license. The 7B model was made available in June 2023 after the initial 3B release in May.

Training ran on 3,072 V100 GPUs through the INCITE program with support from Oak Ridge, MILA, LAION, and EleutherAI. Base, instruct, and chat variants were published, giving developers a genuinely permissive foundation for commercial fine-tunes at a time when most alternatives carried research-only licenses.