KServe
Standardized model inference on Kubernetes — CNCF incubating.
About
KServe is the standardized inference platform for Kubernetes: deploy generative (vLLM, llm-d backends, OpenAI-compatible protocol) and predictive (TensorFlow, PyTorch, sklearn, XGBoost, ONNX) models with autoscaling, canary rollouts, inference graphs, and explainability — a CNCF incubating project. 6.1k stars, Apache-2.0. It's the 'we run on Kubernetes and need model serving done right' answer. The price is the platform itself: KServe assumes you're committed to K8s.
What it can do
- **Generative + predictive**: one platform for LLMs and classical ML
- **vLLM/llm-d backends**: optimized LLM serving
- **OpenAI-compatible**: standard inference protocol
- **Advanced deploys**: canary, ensembles, inference graphs
- **Autoscaling**: including scale-to-zero
Setup
```bash
# K8s cluster required; install via the quickstart:
# https://kserve.github.io/website/docs/get_started/
# then: kubectl apply -f inference-service.yaml
```
Docs: kserve.github.io/website.
Prerequisites
A Kubernetes cluster. KServe installed via its quickstart.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: 6,105 GitHub stars (log-scaled vs catalog peers); last push 0d ago; 18 releases/12mo; 214 open issues. Manual review: CNCF incubating inference on K8s (vLLM/llm-d, OpenAI-compatible), 6.1k stars; K8s-required; smaller community than raw vLLM. License: Apache-2.0.
★ 6,105 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.2/10
- Maintenance · recency, releases, issue load
- 8.6/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · 6,105 GitHub stars (log-scaled vs catalog peers)
- · last push 0d ago; 18 releases/12mo; 214 open issues
- · manual review: CNCF incubating inference on K8s (vLLM/llm-d, OpenAI-compatible), 6.1k stars; K8s-required; smaller community than raw vLLM
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.