Text Generation Inference
HF's inference server that powered HuggingChat — now in maintenance mode.
Retired
Text Generation Inference appears to be discontinued or shut down — archived Mar 2026; superseded by vLLM/SGLang This page is kept as an archive record and is excluded from search engines.
About
Text Generation Inference (TGI) was Hugging Face's Rust+Python+gRPC server for text-generation inference — the engine behind HuggingChat, the Inference API, and Inference Endpoints, pioneering transformers-native optimized architectures. 10.9k stars, Apache-2.0. Status: ARCHIVED and in maintenance mode (March 2026); the README explicitly recommends vLLM or SGLang going forward. Included as a historical entry — do not start new projects on it.
What it can do
- **Optimized inference**: continuous batching, quantization, speculative decoding
- **transformers-native**: architectures adopted by vLLM/SGLang
- **Production pedigree**: powered HuggingChat and HF Inference Endpoints
- **Historical**: the reference the field learned from
Setup
```bash
# Historical — prefer vLLM or SGLang for new deployments
docker run --gpus all -p 8080:80 -v $PWD/data:/data ghcr.io/huggingface/text-generation-inference:latest --model-id <model>
```
Docs: huggingface.github.io/text-generation-inference.
Prerequisites
Docker + NVIDIA GPU. (Prefer vLLM/SGLang for new work.)
Use cases
Works with
Trust score — 6/10
Scored 6/10 (verified tier, metrics-v1). Evidence: 10,880 GitHub stars (log-scaled vs catalog peers); repository is archived — no future fixes expected; capped at 6: archived: maintenance mode. Manual review: powered HuggingChat/Inference API (10.9k stars); ARCHIVED, maintenance mode — README recommends vLLM/SGLang; historical entry only. License: Apache-2.0.
★ 10,880 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.6/10
- Maintenance · recency, releases, issue load
- 2.0/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 6.0/10
- · 10,880 GitHub stars (log-scaled vs catalog peers)
- · repository is archived — no future fixes expected
- · manual review: powered HuggingChat/Inference API (10.9k stars); ARCHIVED, maintenance mode — README recommends vLLM/SGLang; historical entry only
- · capped at 6: archived: maintenance mode
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.