vLLM Production Stack
Reference Helm stack for running vLLM inference clusters on Kubernetes.
About
vLLM Production Stack is the vLLM project's reference system for Kubernetes-native, cluster-wide LLM serving. It packages the vLLM serving engine with intelligent routing, observability, and autoscaling primitives so teams can move from a single GPU pod to production inference infrastructure.
Deployment is via Helm charts with a minimal example values file, exposing the same OpenAI-compatible API as vLLM through a Kubernetes service. The stack includes Prometheus and Grafana observability, a router layer with plans for KV-cache-aware routing and disaggregated prefill, and community-driven performance optimization guides. It is the community's canonical answer to productionizing vLLM at scale.
What it can do
- Helm-based vLLM deployment on Kubernetes
- OpenAI-compatible API through K8s services
- Prometheus and Grafana observability
- Router with prefix-cache-aware routing plans
- Autoscaling based on vLLM-specific metrics
- S3 weight loading and custom chat templates
Setup
Clone the repo, add the Helm repo, and install with a values file for your model and GPU node pool.
Prerequisites
A Kubernetes cluster with GPU nodes and Helm 3.
Use cases
Works with
Trust score — 6/10
Scored 6/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 91; Trending score 64/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.