GritLM
One 7B model that does both jobs: bidirectional attention for embeddings, causal attention for generation.
About
GritLM (Generative Representational Instruction Tuning) is the model that asked why you should train two models when one can do both. Built by researchers at Contextual AI and released in February 2024, the 7B model runs bidirectional attention when it needs to embed and causal attention when it needs to generate, switching modes with a single keyword argument. It is fine-tuned from Mistral-7B and released under Apache 2.0.
The repo reports 66.8 average on MTEB for embedding and 55.5 on generative benchmarks for GritLM-7B, matching a dedicated embedding-only variant while adding generation almost for free. The idea has been hugely influential: it underpins much of the current wave of decoder-based embedding models, even though serving one 7B model for both roles is heavier than most retrieval pipelines need.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- MTEB average: 66.8 embedding / 55.5 generative for GritLM-7B (author-reported, repo README)
- GritLM-8x7B: 65.7 / 65.7 (author-reported)
- Embedding-only variant matches 66.8; generative-only variant collapses to 41.2 on MTEB (author-reported ablations)
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: HF API confirms Apache 2.0 license and Feb 2024 release with ~25.7K downloads; the 66.8 MTEB figure is in the authors' public repo README with ablation variants. Strengths: peer-reviewed paper, fully open, hugely influential method, reproducible recipe. Docked for: (1) benchmark figures are author-run with no independent replication found; (2) 7B size is heavy for pure retrieval duty, so it is more influential than deployed.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 6.0/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · HF API confirms Apache 2.0 and Feb 2024 release; ~25.7K downloads
- · MTEB 66.8 (emb) / 55.5 (gen) from the authors' public repo README with ablation variants
- · Peer-reviewed paper; unified embedding+generation method hugely influential on decoder-embedding work