SaulLM-54B

SaulLM-54B is the mid-size model in Equall's SaulLM family of legal LLMs, built with CentraleSupélec and CINES and released in 2024. It starts from the Mixtral architecture and undergoes large-scale domain adaptation: continued pretraining on more than 540 billion legal tokens, then a legal instruction-tuning protocol and preference alignment on synthetic legal data. The family also includes a 141B model and a smaller 7B, with base, instruct and aligned variants of each.

The paper reports state-of-the-art results among open models on LegalBench-Instruct, outperforming GPT-4, Llama 3 and the original Mixtral on legal reasoning tasks, and the whole release is MIT licensed, which is unusually permissive for a domain model of this size. The authors are candid about the tradeoff: the models are weaker than Mixtral-Instruct and Llama 3 at following generic instructions. For legal document work, contract analysis and case-law reasoning, it is one of the few open models with serious legal pretraining behind it; for general chat, use something else.

• Authors report SOTA among open models on LegalBench-Instruct, outperforming GPT-4, Llama 3 and Mixtral on legal reasoning (provider claim, arXiv:2407.19584)
• Authors note weaker generic instruction-following than Mixtral-Instruct and Llama 3 (paper's own limitation)
• No independent benchmark table verified