LangDB
Open-source AI gateway built in Rust for LLM traffic.
About
LangDB AI Gateway is a high-performance, open-source API gateway for managing and monitoring LLM traffic at scale. Built in Rust, it sits between clients and backend LLMs with caching, prompt de-duplication, rate limiting, latency-aware routing, and built-in metrics, logging, and request tracing.
It supports OpenAI, Anthropic, and local models, integrates with ClickHouse for analytics (including calling LangDB APIs as UDFs inside ClickHouse), and offers a hosted version at langdb.ai alongside the self-hostable core. The project also ships MCP server support for tool-aware routing.
What it can do
- Open-source Rust AI gateway
- Caching and latency-aware routing
- MCP server support
- ClickHouse-native analytics and tracing
Setup
Cargo install or Docker: configure config.yaml and run the ai-gateway binary; hosted version at langdb.ai.
Prerequisites
Provider API keys; optional ClickHouse for analytics.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 105; Trending score 76/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.