LangWatch
Open-source LLM monitoring, evals, and agent simulations.
About
LangWatch is an open-source platform for LLM evaluations and AI agent testing, built OpenTelemetry-native so it works with any framework or provider. It combines production monitoring, offline evals, end-to-end agent simulations, and an AI gateway with budgets and guardrails, plus a GitHub integration that keeps prompts in version control.
The free tier covers 200K events a month and the codebase is Apache 2.0. Being newer than the biggest names, its library of templates and community examples is still catching up.
What it can do
- OTel-native tracing and monitoring
- Offline and online LLM evaluations
- End-to-end agent simulations
- AI gateway with budgets and guardrails
- Prompt versioning via GitHub
- Annotations and failure queues
Setup
- Create an account at langwatch.ai or self-host.
- Install the SDK for your language.
- Send traces via the OpenTelemetry integration.
- Build evals and agent simulations from production traces.
Prerequisites
An LLM integration to instrument; a LangWatch account or self-hosted instance.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Apache 2.0, self-hostable; evals + monitoring + agent simulations in one loop; free tier 200K events/month; newer; template library still growing. License: Apache 2.0.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.0/10
- Maintenance · recency, releases, issue load
- 8.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Apache 2.0, self-hostable
- · evals + monitoring + agent simulations in one loop
- · free tier 200K events/month
- · newer; template library still growing
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.