# MLX LM review

> MLX LM is a Python package for generating text with and fine-tuning large language models on Apple silicon using MLX.

Devtoolwatch Board, the week of Oct 25–31, 2026. Canonical page: https://index.devtoolwatch.com/tools/mlx-lm/

## TL;DR

MLX LM is a Python package for running and fine-tuning large language models on Apple silicon with MLX. It is for developers who want local LLM inference and training on Macs. MLX LM pulls models from the Hugging Face Hub with a single command and can quantize models and upload them back.

## MLX LM at a glance

- **Category:** [Inference and training](https://index.devtoolwatch.com/categories/inference-and-training/)
- **Maintained by:** ml-explore (organization, [@ml-explore](https://github.com/ml-explore) on GitHub)
- **License:** MIT
- **Language:** Python
- **Repository:** [github.com/ml-explore/mlx-lm](https://github.com/ml-explore/mlx-lm)
- **PyPI package:** [mlx-lm](https://pypi.org/project/mlx-lm/) 0.32.2
- **Latest release:** [v0.32.2](https://github.com/ml-explore/mlx-lm/releases/tag/v0.32.2) on GitHub, Oct 16, 2026
- **First published:** Mar 11, 2025

## What MLX LM does

- MLX LM generates text and runs an interactive chat REPL that keeps chat context for the lifetime of the session.
- MLX LM loads MLX-compatible models from the Hugging Face Hub, including thousands in the MLX Community organization.
- MLX LM quantizes models, for example to 4-bit, and can upload converted models to the Hugging Face Hub.
- MLX LM supports low-rank and full model fine-tuning, including on quantized models.
- MLX LM offers distributed inference and fine-tuning with mx.distributed.
- MLX LM handles long contexts with a rotating key-value cache, prompt caching, and a configurable prefill step size.
- MLX LM provides a Python API with streaming generation, batch generation, custom samplers, and logits processors.

### How it works

MLX LM runs as command-line tools such as mlx_lm.generate, mlx_lm.chat, mlx_lm.convert, and mlx_lm.server, or as a Python module with load, generate, stream_generate, and convert functions. MLX LM downloads models from the Hugging Face Hub and runs them with MLX on Apple silicon. Converted models are saved to the mlx_model path by default.

## How to install MLX LM

**With pip**

```
pip install mlx-lm
```

**With conda**

```
conda install -c conda-forge mlx-lm
```

### Requirements

- Apple silicon Mac
- Python 3.11 or higher
- macOS 15.0 or higher for wiring memory to speed up large models

### First steps

1. Generate text with the default model: mlx_lm.generate --prompt "How tall is Mt Everest?"
2. Start an interactive chat session with mlx_lm.chat.
3. Pick a different model with the --model flag, such as any model from the MLX Community organization.
4. Run mlx_lm.generate -h to list options for the model, sampling parameters, and more.

## Who MLX LM is for

- Developers running LLMs locally on Apple silicon Macs
- Developers fine-tuning LLMs with LoRA or full fine-tuning on a Mac
- Developers quantizing Hugging Face models and publishing MLX versions
- Python developers who want a scriptable generation API with streaming and custom sampling

### Limitations

- Models that are large relative to the machine's RAM can be slow, and the memory wiring speedup needs macOS 15 or higher.
- Small rotating key-value cache sizes save RAM but lower output quality.
- Some models need trust_remote_code enabled for their tokenizer.

## Usage data (the week of Oct 25–31, 2026)

- Board rank: #320 overall, #39 in Inference and training, momentum score 28
- Downloads that week: 137k (−1% vs 4-week average)
- New GitHub stars per week (last 12 weeks): Aug 9: 89; Aug 16: 115; Aug 23: 91; Aug 30: 68; Sep 6: 90; Sep 13: 82; Sep 20: 85; Sep 27: 57; Oct 4: 58; Oct 11: 127; Oct 18: 84; Oct 25: 55

## MLX LM alternatives

| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | [dsh-our-free-model](https://index.devtoolwatch.com/tools/ebony-vinyl-dsh-our-free-model/) | 54.3k | – |
| #2 | [magpie](https://index.devtoolwatch.com/tools/yetone-magpie/) | 26.9k | – |
| #3 | [Strata](https://index.devtoolwatch.com/tools/niko1221-strata/) | 37.1k | – |
| #4 | [edge0](https://index.devtoolwatch.com/tools/edge0-ai-edge0/) | 8.9k | – |
| #5 | [OmniRoute](https://index.devtoolwatch.com/tools/diegosouzapw-omniroute/) | 83.2k | 106k |
| #6 | [CLIProxyAPI](https://index.devtoolwatch.com/tools/router-for-me-cliproxyapi/) | 57.1k | – |

## MLX LM FAQ

### Is MLX LM free and open source?

Yes. MLX LM's source code is on GitHub (ml-explore/mlx-lm) under the MIT license, so you can use it for free.

### How do I install MLX LM?

Run `pip install mlx-lm`. The mlx-lm package needs Python 3.11 or newer. The install section above has the other options from the project's README.

### Is MLX LM still maintained?

Yes. The ml-explore/mlx-lm repository was last updated on Oct 20, 2026, and the latest release, v0.32.2 on GitHub, came out on Oct 16, 2026.

### How popular is MLX LM?

MLX LM has 7.5k GitHub stars as of Nov 1, 2026. It gained 55 new stars in the week of Oct 25–31, 2026, against a 12-week average of 83 a week. Its main package had 137k downloads that week, −1% against its own 4-week average. That made it #39 among LLM inference and training tools on the Devtoolwatch Board.

### MLX LM vs Ollama: how do they compare?

MLX LM and Ollama are both tracked by Devtoolwatch as LLM inference and training tools. MLX LM has 7.5k GitHub stars and Ollama has 185.5k. In the week of Oct 25–31, 2026, MLX LM's main package had 137k downloads and Ollama's had 4M. Ollama is a tool for running and building with open AI models locally or in the cloud on macOS, Windows, and Linux. [Ollama review](https://index.devtoolwatch.com/tools/ollama/)

### MLX LM vs vLLM: how do they compare?

MLX LM and vLLM are both tracked by Devtoolwatch as LLM inference and training tools. MLX LM has 7.5k GitHub stars and vLLM has 96k. In the week of Oct 25–31, 2026, MLX LM's main package had 137k downloads and vLLM's had 677.2k. vLLM is an open-source Python library and engine for high-throughput, memory-efficient inference and serving of large language models. [vLLM review](https://index.devtoolwatch.com/tools/vllm/)

### What are the best alternatives to MLX LM?

Other LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: dsh-our-free-model (#1), magpie (#2), Strata (#3), edge0 (#4), OmniRoute (#5). The alternatives table above has their numbers.

### What is MLX LM?

MLX LM is a Python package from the ml-explore organization for generating text and fine-tuning large language models on Apple silicon with MLX. MLX LM integrates with the Hugging Face Hub and supports quantization, LoRA and full fine-tuning, and distributed inference.

### Where does MLX LM store converted models?

MLX LM saves converted models to the path mlx_model by default when you use convert. MLX LM downloads source models from the Hugging Face Hub. The sources do not state the download cache location.

### Which models can MLX LM run?

MLX LM runs any MLX-compatible model passed with the --model flag. Thousands are available in the MLX Community Hugging Face organization. The default model is mlx-community/Llama-3.2-3B-Instruct-4bit. If a model is not supported, you can file an issue or submit a pull request.

### How do I speed up repeated queries on a long context in MLX LM?

MLX LM can cache a prompt with mlx_lm.cache_prompt and reuse it through --prompt-cache-file in mlx_lm.generate. The cached prompt is treated as a prefix, and the model is read from the cache. Prompt caching also works in the Python API.

### Does MLX LM include a server?

MLX LM includes an mlx_lm.server command, which accepts options such as --prefill-step-size. Release notes mention fixes to tool call handling in the server. The README does not document server usage in detail.

## Sources

1. [ml-explore/mlx-lm on GitHub](https://github.com/ml-explore/mlx-lm)
2. [MLX LM README](https://github.com/ml-explore/mlx-lm/blob/main/README.md)
3. [mlx-lm on PyPI](https://pypi.org/project/mlx-lm/)
4. [Release v0.32.2 (Oct 16, 2026)](https://github.com/ml-explore/mlx-lm/releases/tag/v0.32.2)
5. [Release v0.32.1 (Oct 15, 2026)](https://github.com/ml-explore/mlx-lm/releases/tag/v0.32.1)
6. [Release v0.32.0 (Oct 15, 2026)](https://github.com/ml-explore/mlx-lm/releases/tag/v0.32.0)
7. [Devtoolwatch Board, the week of Oct 25–31, 2026](https://index.devtoolwatch.com/)