Text Embeddings Inference
Blazing-fast embedding and reranker inference server.
About
Text Embeddings Inference (TEI) is Hugging Face's Rust-powered server for embedding and reranker models: high-throughput, low-latency inference for the models behind every RAG pipeline. 5.1k stars, Apache-2.0, actively maintained. If you self-host embeddings (bge, e5, jina) instead of paying per-token API fees, TEI is the standard way to serve them — Docker one-liner, OpenAI-compatible endpoints. Embeddings and reranking only; it's not a general LLM server.
What it can do
- **High throughput**: Rust + Candle, built for embedding scale
- **Rerankers**: sequence-classification reranking support
- **SOTA models**: bge, e5, jina, and more out of the box
- **OpenAI-compatible**: drop-in /embed endpoints
- **Quantization**: efficient deployment options
Setup
```bash
docker run --gpus all -p 8080:80 -v $PWD/data:/data ghcr.io/huggingface/text-embeddings-inference:latest --model-id BAAI/bge-large-en-v1.5
```
Then POST to /embed. Docs: huggingface.github.io/text-embeddings-inference.
Prerequisites
Docker; GPU recommended for larger embedding models.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: 5,076 GitHub stars (log-scaled vs catalog peers); last push 6d ago; 6 releases/12mo; 216 open issues. Manual review: HF's Rust embedding/reranker server, 5.1k stars, active; embeddings-only; smaller mindshare than LLM serving. License: Apache-2.0.
★ 5,076 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.1/10
- Maintenance · recency, releases, issue load
- 7.7/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · 5,076 GitHub stars (log-scaled vs catalog peers)
- · last push 6d ago; 6 releases/12mo; 216 open issues
- · manual review: HF's Rust embedding/reranker server, 5.1k stars, active; embeddings-only; smaller mindshare than LLM serving
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.