Devtoolwatch BoardGet it every Monday

Arize Phoenix review

Arize Phoenix is an open-source, self-hosted AI observability platform for tracing, evaluating, and troubleshooting LLM applications and agents.

Updated Nov 1, 2026. Review written Oct 22, 2026 from the project's own sources; usage data refreshed every Monday.

TL;DR

Arize Phoenix is Arize's open-source AI observability platform for experimentation, evaluation, and troubleshooting. It is for developers building LLM applications and agents who want to trace runs, score outputs, and test prompt changes. Phoenix is built on OpenTelemetry and OpenInference, and it is vendor and language agnostic.

Arize Phoenix at a glance

Category
Evals and observability
Maintained by
Arize AI (organization, @Arize-ai on GitHub)
Language
Python, TypeScript, Jupyter Notebook
Repository
github.com/Arize-ai/phoenix
Website
arize.com/docs/phoenix
PyPI package
arize-phoenix 20.24.0
Latest release
arize-phoenix-v20.24.0 on GitHub, Oct 21, 2026
First published
Nov 9, 2022

What Arize Phoenix does

  • Arize Phoenix traces LLM application runtime with OpenTelemetry-based instrumentation, capturing model calls, retrieval, tool use, and custom logic.
  • Arize Phoenix scores traces and spans with LLM-based evaluators, code-based checks, or human labels.
  • Arize Phoenix creates versioned datasets of examples for experimentation, evaluation, and fine-tuning.
  • Arize Phoenix runs experiments that track and evaluate changes to prompts, LLMs, and retrieval on the same inputs.
  • Arize Phoenix includes a playground to optimize prompts, compare models, adjust parameters, and replay traced LLM calls.
  • Arize Phoenix manages prompt changes with version control, tagging, and experimentation.
  • Arize Phoenix exposes a remote MCP server at `/mcp` so Claude Code, Cursor, and other MCP clients can query traces, datasets, and experiments.

How it works

Arize Phoenix runs as a server on your local machine, in a container, or in the cloud. Your application sends traces to it over OpenTelemetry (OTLP), usually through OpenInference auto-instrumentation for frameworks like LangChain, LlamaIndex, DSPy, and Vercel AI SDK. Lightweight Python and TypeScript packages provide OpenTelemetry wrappers, a REST API client, and eval tooling.

How to install and start using Arize Phoenix

With pip

pip install arize-phoenix

With uvx (no install)

uvx arize-phoenix serve

Instrument your app via CLI

npx @arizeai/phoenix-cli setup

Requirements

  • Python 3.11 or later, below 3.15, for the arize-phoenix package.
  • Node.js with npx to use the @arizeai/phoenix-cli setup command.

First steps

  1. Start the Phoenix server with `phoenix serve` after installing the arize-phoenix package.
  2. From your project directory, run the Phoenix CLI setup to detect your framework and LLM provider and install OpenInference instrumentation.
  3. Run your application and inspect the exported traces in the Phoenix UI.
  4. Attach evaluators to a dataset so they run automatically during experiments.

Who Arize Phoenix is for

  • Developers debugging LLM apps and agents built with LangChain, LlamaIndex, DSPy, CrewAI, or Vercel AI SDK.
  • Teams that want self-hosted observability built on OpenTelemetry standards.
  • Developers evaluating RAG relevance and answer relevance in LLM applications.
  • Developers who use coding agents like Claude Code or Cursor and want them to query traces.

Limitations to know

  • Arize Phoenix is self-hosted. A managed platform is offered separately as Arize AX.
  • The Azure deployment template serves plain HTTP and needs a TLS proxy before production use.
  • The TypeScript evals package, @arizeai/phoenix-evals, is an alpha release.
  • The standalone stdio MCP server package is in maintenance mode, superseded by the built-in remote MCP server.

Is Arize Phoenix growing? Usage data

In the week of Oct 25–31, 2026, Arize Phoenix ranked #219 on the Devtoolwatch Board and #10 in Evals and observability, with a momentum score of 42 out of 100. Its main package had 158.1k downloads that week, +2% against its own 4-week average.

New GitHub stars per week for Arize Phoenix12 weeks from Aug 9, 2026: 117, 86, 102, 102, 93, 113, 91, 79, 91, 110, 106, 101 new stars, an average of 99 a week.Aug 9week of Oct 25
New GitHub stars per week for the last 12 weeks, from GitHub's stargazer history (today's stargazers by the week they starred).
Show the weekly numbers
New GitHub stars per week for Arize Phoenix
Week ofNew stars
Aug 9, 2026117
Aug 16, 202686
Aug 23, 2026102
Aug 30, 2026102
Sep 6, 202693
Sep 13, 2026113
Sep 20, 202691
Sep 27, 202679
Oct 4, 202691
Oct 11, 2026110
Oct 18, 2026106
Oct 25, 2026101

On Hacker News

The Board ranks tools by four signals each week: new stars, new stars for the repo's size, added downloads and download growth beyond the market. How the ranking works.

Arize Phoenix alternatives

Other LLM evals and observability tools on the Board in the week of Oct 25–31, 2026, by momentum:

Evals and observability ranked by momentum, the week of Oct 25–31, 2026
RankToolStarsDownloads/wk
#1SkillSpectorSkillSpector is NVIDIA's security scanner that checks AI agent skills for vulnerabilities, malicious patterns, and supply-chain risks…23.7k–
#2JudgevalJudgeval is an open-source Python SDK for tracing and evaluating LLM-powered agents with prompt-based judges and online monitoring.1.1k80.2k
#3A.I.G (AI-Infra-Guard)A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak…7.2k–
#4LangfuseLangfuse is an open source LLM engineering platform for tracing, evaluating, prompt management, and debugging LLM applications and agents.36.8k9.8M
#5OpikOpik is an open-source platform from Comet for tracing, evaluating, and monitoring LLM applications, RAG systems, and AI agents.23.1k709.1k
#6promptfoopromptfoo is a CLI and library for evaluating and red-teaming LLM-based apps, including prompts, agents, and RAG pipelines.26.7k1.2M

All LLM evals and observability tools, ranked

Recent Arize Phoenix releases

Arize Phoenix FAQ

Is Arize Phoenix free and open source?

Arize Phoenix's code is on GitHub (Arize-ai/phoenix), but the repository doesn't declare a standard open-source license. Check its license file before using it commercially.

How do I install Arize Phoenix?

Run pip install arize-phoenix. The arize-phoenix package needs Python 3.11 or newer, below 3.15. The install section above has the other options from the project's README.

Is Arize Phoenix still maintained?

Yes. The Arize-ai/phoenix repository was last updated on Oct 21, 2026, and the latest release, arize-phoenix-v20.24.0 on GitHub, came out on Oct 21, 2026.

How popular is Arize Phoenix?

Arize Phoenix has 12.1k GitHub stars as of Nov 1, 2026. It gained 101 new stars in the week of Oct 25–31, 2026, against a 12-week average of 99 a week. Its main package had 158.1k downloads that week, +2% against its own 4-week average. That made it #10 among LLM evals and observability tools on the Devtoolwatch Board.

Arize Phoenix vs Langfuse: how do they compare?

Arize Phoenix and Langfuse are both tracked by Devtoolwatch as LLM evals and observability tools. Arize Phoenix has 12.1k GitHub stars and Langfuse has 36.8k. In the week of Oct 25–31, 2026, Arize Phoenix's main package had 158.1k downloads and Langfuse's had 9.8M. Langfuse is an open source LLM engineering platform for tracing, evaluating, prompt management, and debugging LLM applications and agents. Langfuse review.

Arize Phoenix vs MLflow: how do they compare?

Arize Phoenix and MLflow are both tracked by Devtoolwatch as LLM evals and observability tools. Arize Phoenix has 12.1k GitHub stars and MLflow has 28.6k. In the week of Oct 25–31, 2026, Arize Phoenix's main package had 158.1k downloads and MLflow's had 5.2M. MLflow is an open source AI engineering platform for tracing, evaluating, monitoring, and managing agents, LLM applications, and ML models. MLflow review.

What are the best alternatives to Arize Phoenix?

Other LLM evals and observability tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: SkillSpector (#1), Judgeval (#2), A.I.G (AI-Infra-Guard) (#3), Langfuse (#4), Opik (#5). The alternatives table above has their numbers.

What is Arize Phoenix?

Arize Phoenix is Arize's open-source AI observability platform for experimentation, evaluation, and troubleshooting of AI applications. It provides tracing, evaluation, datasets, experiments, a prompt playground, and prompt management.

What is Arize Phoenix used for?

Arize Phoenix is used to debug and improve AI applications. You send traces to see what happened in a run, score outputs to find failures and regressions, iterate on prompts with real production examples, and compare changes with experiments.

How does Arize Phoenix work?

Arize Phoenix is built on OpenTelemetry and powered by OpenInference instrumentation. It accepts traces over OTLP and auto-instruments frameworks and providers such as LangChain, LlamaIndex, OpenAI, Anthropic, and Bedrock, in Python, TypeScript, and Java.

What is the difference between Arize Phoenix and Arize AX?

Arize Phoenix is open-source and self-hosted. Arize AX is Arize's managed enterprise platform built on the same open standards. The same OpenTelemetry and OpenInference instrumentation works with both.

Can I use my own evaluators with Arize Phoenix?

Arize Phoenix supports pre-built and custom LLM-based evaluators. It also accepts evaluators from Ragas, Deepeval, or Cleanlab, along with code-based checks and human annotations added in the UI.

Sources

  1. Arize-ai/phoenix on GitHub
  2. Arize Phoenix README
  3. arize-phoenix on PyPI
  4. Arize Phoenix website
  5. Release arize-phoenix-v20.24.0 (Oct 21, 2026)
  6. Release @arizeai/[email protected] (Oct 21, 2026)
  7. Release @arizeai/[email protected] (Oct 21, 2026)
  8. Hacker News: Show HN: Phoenix OSS – Applying LLM Spans, Traces, and Evals for AI Insights
  9. Devtoolwatch Board, the week of Oct 25–31, 2026

Arize Phoenix README badge

Maintainers can show the tool's weekly Board rank in their README:

Devtoolwatch badge for Arize Phoenix

[![Devtoolwatch](https://index.devtoolwatch.com/badge/phoenix.svg)](https://index.devtoolwatch.com/tools/phoenix/)

How this page is made: Devtoolwatch writes each review from the project's README, releases, package metadata and website, with an AI model, then checks every claim against the source it cites and drops what the sources don't support. The numbers come from Devtoolwatch's daily GitHub snapshots, GitHub's stargazer history, and the npm and PyPI download APIs. Spotted an error? Email [email protected].