Japanese StableLM 7B

Japanese StableLM Alpha was released by Stability AI on August 10, 2023, the company's first non-English language model. The 7-billion-parameter model was trained on 750 billion tokens of Japanese and English web text using an extended GPT-NeoX stack with SwiGLU and xPos, plus datasets built with Stability AI Japan's community and the EleutherAI Polyglot Japanese team. Stability AI reported it as the top-scoring publicly available Japanese LM across eight lm-evaluation-harness tasks.

Two variants shipped: Japanese StableLM Base Alpha 7B under Apache 2.0 for commercial use, and an Instruct Alpha 7B for research only. Follow-ups included Japanese Stable LM 3B-4E1T and Gamma 7B.