# MLflow review

> MLflow is an open source AI engineering platform for tracing, evaluating, monitoring, and managing agents, LLM applications, and ML models.

Devtoolwatch Board, the week of Oct 25–31, 2026. Canonical page: https://index.devtoolwatch.com/tools/mlflow/

## TL;DR

MLflow is an open source platform for debugging, evaluating, monitoring, and optimizing agents, LLM applications, and ML models. It is for teams that ship AI to production. MLflow combines OpenTelemetry-based tracing, LLM judges, prompt management, an AI Gateway, and classic ML experiment tracking and model registry in one vendor-neutral tool.

## MLflow at a glance

- **Category:** [Evals and observability](https://index.devtoolwatch.com/categories/evals-and-observability/)
- **Maintained by:** MLflow (organization, [@mlflow](https://github.com/mlflow) on GitHub)
- **License:** Apache 2.0
- **Language:** Python, TypeScript, JavaScript
- **Repository:** [github.com/mlflow/mlflow](https://github.com/mlflow/mlflow)
- **Website:** [mlflow.org](https://mlflow.org)
- **PyPI package:** [mlflow](https://pypi.org/project/mlflow/) 3.17.2
- **Latest release:** [mlflow 3.17.2](https://pypi.org/project/mlflow/) on PyPI, Oct 21, 2026
- **First published:** Jun 5, 2018

## What MLflow does

- MLflow captures complete traces of LLM applications and agents, built on OpenTelemetry, with one-line automatic tracing for more than 60 frameworks.
- MLflow runs systematic evaluations using 50+ built-in metrics and LLM judges, or custom ones, to catch regressions before production.
- MLflow versions, tests, and deploys prompts with lineage tracking, and can automatically optimize prompts.
- MLflow's AI Gateway routes requests across LLM providers through an OpenAI-compatible interface with rate limits, fallbacks, guardrails, and traffic splitting.
- MLflow tracks experiments, parameters, and metrics, and manages model lifecycles in a Model Registry.
- MLflow deploys models to batch and real-time scoring on Docker, Kubernetes, Azure ML, AWS SageMaker, and more.
- MLflow Agent Server hosts agents on FastAPI with request validation, streaming support, and built-in tracing.

### How it works

MLflow runs as a tracking server, for example started with uvx, that your code logs to through the Python SDK or autologging. Traces and metrics appear in the MLflow UI at localhost:5000. MLflow supports Python, TypeScript/JavaScript, and Java, integrates with OpenTelemetry and MCP, and runs locally, on-premises, on cloud platforms, or on managed services like Databricks and SageMaker.

## How to install MLflow

**Setup wizard**

```
curl -LsSf https://mlflow.org/wizard/setup.sh | sh
```

**Start server with uvx**

```
uvx mlflow server
```

### Requirements

- Python 3.10 or later for the mlflow package.
- The setup wizard requires a Git repository and Claude Code, Codex, or OpenCode installed.

### First steps

1. Start the MLflow server with uvx mlflow server.
2. In Python, call mlflow.set_tracking_uri("http://localhost:5000") and mlflow.openai.autolog() to enable logging.
3. Run your OpenAI client code as usual so calls are traced.
4. Explore traces and metrics in the MLflow UI at http://localhost:5000.

## Who MLflow is for

- Teams debugging and monitoring LLM applications and agents in production.
- Developers evaluating LLM output quality with built-in or custom LLM judges.
- ML teams tracking experiments and managing models through a registry.
- Organizations wanting a self-hostable, vendor-neutral platform across cloud providers.

### Limitations

- Some features, such as review queues and Unity Catalog traces, are specific to Databricks.

## Usage data (the week of Oct 25–31, 2026)

- Board rank: #165 overall, #8 in Evals and observability, momentum score 49
- Downloads that week: 5.2M (+11% vs 4-week average)
- New GitHub stars per week (last 12 weeks): Aug 9: 110; Aug 16: 98; Aug 23: 122; Aug 30: 94; Sep 6: 118; Sep 13: 119; Sep 20: 100; Sep 27: 110; Oct 4: 102; Oct 11: 101; Oct 18: 98; Oct 25: 103

## MLflow alternatives

| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | [SkillSpector](https://index.devtoolwatch.com/tools/nvidia-skillspector/) | 23.7k | – |
| #2 | [Judgeval](https://index.devtoolwatch.com/tools/judgmentlabs-judgeval/) | 1.1k | 80.2k |
| #3 | [A.I.G (AI-Infra-Guard)](https://index.devtoolwatch.com/tools/tencent-ai-infra-guard/) | 7.2k | – |
| #4 | [Langfuse](https://index.devtoolwatch.com/tools/langfuse/) | 36.8k | 9.8M |
| #5 | [Opik](https://index.devtoolwatch.com/tools/opik/) | 23.1k | 709.1k |
| #6 | [promptfoo](https://index.devtoolwatch.com/tools/promptfoo/) | 26.7k | 1.2M |

## MLflow FAQ

### Is MLflow free and open source?

Yes. MLflow's source code is on GitHub (mlflow/mlflow) under the Apache 2.0 license, so you can use it for free.

### How do I install MLflow?

Run `curl -LsSf https://mlflow.org/wizard/setup.sh | sh`. The mlflow package needs Python 3.10 or newer. The install section above has the other options from the project's README.

### Is MLflow still maintained?

Yes. The mlflow/mlflow repository was last updated on Oct 25, 2026, and the latest release, mlflow 3.17.2 on PyPI, came out on Oct 21, 2026.

### How popular is MLflow?

MLflow has 28.6k GitHub stars as of Nov 1, 2026. It gained 103 new stars in the week of Oct 25–31, 2026, against a 12-week average of 106 a week. Its main package had 5.2M downloads that week, +11% against its own 4-week average. That made it #8 among LLM evals and observability tools on the Devtoolwatch Board.

### MLflow vs Langfuse: how do they compare?

MLflow and Langfuse are both tracked by Devtoolwatch as LLM evals and observability tools. MLflow has 28.6k GitHub stars and Langfuse has 36.8k. In the week of Oct 25–31, 2026, MLflow's main package had 5.2M downloads and Langfuse's had 9.8M. Langfuse is an open source LLM engineering platform for tracing, evaluating, prompt management, and debugging LLM applications and agents. [Langfuse review](https://index.devtoolwatch.com/tools/langfuse/)

### What are the best alternatives to MLflow?

Other LLM evals and observability tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: SkillSpector (#1), Judgeval (#2), A.I.G (AI-Infra-Guard) (#3), Langfuse (#4), Opik (#5). The alternatives table above has their numbers.

### What is MLflow?

MLflow is an open source AI engineering platform for agents, LLMs, and ML models. It covers tracing, evaluation, prompt management, an AI Gateway, experiment tracking, model registry, and deployment.

### Is MLflow a Python library?

MLflow is distributed as a Python package on PyPI requiring Python 3.10 or later. MLflow also supports TypeScript/JavaScript and Java, and runs as a server with a web UI.

### Is MLflow a model registry?

MLflow includes a Model Registry for collaboratively managing the full lifecycle of ML models, alongside experiment tracking, evaluation, and deployment tools.

### How do I use MLflow locally?

MLflow runs locally by starting a server with uvx mlflow server, pointing your code at http://localhost:5000 with mlflow.set_tracking_uri, and enabling autologging. View traces in the UI at that address.

### How do I use MLflow in Databricks?

MLflow can be hosted on Databricks, and the setup wizard can connect to a Databricks workspace. Recent releases add Databricks-specific features such as review queues.

### Can MLflow be used with AWS SageMaker?

MLflow lists Amazon SageMaker as a hosting option and can deploy models to AWS SageMaker for scoring.

## Sources

1. [mlflow/mlflow on GitHub](https://github.com/mlflow/mlflow)
2. [MLflow README](https://github.com/mlflow/mlflow/blob/master/README.md)
3. [mlflow on PyPI](https://pypi.org/project/mlflow/)
4. [MLflow website](https://mlflow.org)
5. [Release model-catalog/latest (Apr 6, 2026)](https://github.com/mlflow/mlflow/releases/tag/model-catalog/latest)
6. [Release v3.17.2 (Oct 21, 2026)](https://github.com/mlflow/mlflow/releases/tag/v3.17.2)
7. [Release v3.17.1 (Oct 13, 2026)](https://github.com/mlflow/mlflow/releases/tag/v3.17.1)
8. [Hacker News: Show HN: MLflow – A platform for the complete machine learning](https://news.ycombinator.com/item?id=17263133)
9. [Devtoolwatch Board, the week of Oct 25–31, 2026](https://index.devtoolwatch.com/)