Arize Phoenix
Open-source LLM tracing and evaluation, OpenTelemetry-native, runs in a notebook.
Acquired — still operating
acquired by Dynatrace Aug 2026
About
Arize Phoenix is an open-source AI observability and evaluation platform built on OpenTelemetry: trace every agent step, run LLM-as-judge evals with built-in evaluators (faithfulness, relevance, hallucination, toxicity), build datasets from traces, and run experiments — all self-hosted with your traces staying in your environment. It works out of the box with LangChain, LangGraph, LlamaIndex, CrewAI, DSPy, and the OpenAI Agents SDK.
It runs locally in a notebook with zero infrastructure (pip install arize-phoenix) or as a deployed server, and Arize offers hosted Phoenix Cloud instances. ~11k GitHub stars, ELv2 licensed.
Honest limits: Arize was acquired by Dynatrace in August 2026, so the product's direction may shift; and its deepest strengths are RAG/agent debugging rather than full production APM.
What it can do
- OTel-native tracing for agents and RAG
- Built-in evaluators (faithfulness, relevance, toxicity)
- Datasets and experiment tracking
- Embedding analysis and drift views
- Prompt playground
- Notebook-local or self-hosted
Setup
- Install: pip install arize-phoenix.
- Launch: phoenix.trace_notebook() or run the server.
- Instrument with @trace decorators or auto-instrumentation for your framework.
- Inspect traces in the UI, then build evals and datasets from them.
Prerequisites
Python 3.9+; an LLM app to instrument.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (metrics tier, metrics-v1). Evidence: ~11k GitHub stars; OpenTelemetry-native tracing; Arize acquired by Dynatrace Aug 2026 — direction may shift; Runs locally in a notebook with zero infrastructure. License: ELv2 licensed, open source.
★ 11,000 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 9.0/10
- Openness · license + docs
- 8.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · ~11k GitHub stars; OpenTelemetry-native tracing
- · Arize acquired by Dynatrace Aug 2026 — direction may shift
- · Runs locally in a notebook with zero infrastructure