Devtoolwatch BoardMLX LM review
MLX LM is a Python package for generating text with and fine-tuning large language models on Apple silicon using MLX.
TL;DR
MLX LM is a Python package for running and fine-tuning large language models on Apple silicon with MLX. It is for developers who want local LLM inference and training on Macs. MLX LM pulls models from the Hugging Face Hub with a single command and can quantize models and upload them back.
- 7.5kGitHub stars
- +55new stars, week of Oct 25
- 137kdownloads a week, −1% vs 4-week avg
- #39in Inference and training this week
MLX LM at a glance
- Category
- Inference and training
- Maintained by
- ml-explore (organization, @ml-explore on GitHub)
- License
- MIT
- Language
- Python
- Repository
- github.com/ml-explore/mlx-lm
- PyPI package
- mlx-lm 0.32.2
- Latest release
- v0.32.2 on GitHub, Oct 16, 2026
- First published
- Mar 11, 2025
What MLX LM does
- MLX LM generates text and runs an interactive chat REPL that keeps chat context for the lifetime of the session.
- MLX LM loads MLX-compatible models from the Hugging Face Hub, including thousands in the MLX Community organization.
- MLX LM quantizes models, for example to 4-bit, and can upload converted models to the Hugging Face Hub.
- MLX LM supports low-rank and full model fine-tuning, including on quantized models.
- MLX LM offers distributed inference and fine-tuning with mx.distributed.
- MLX LM handles long contexts with a rotating key-value cache, prompt caching, and a configurable prefill step size.
- MLX LM provides a Python API with streaming generation, batch generation, custom samplers, and logits processors.
How it works
MLX LM runs as command-line tools such as mlx_lm.generate, mlx_lm.chat, mlx_lm.convert, and mlx_lm.server, or as a Python module with load, generate, stream_generate, and convert functions. MLX LM downloads models from the Hugging Face Hub and runs them with MLX on Apple silicon. Converted models are saved to the mlx_model path by default.
How to install and start using MLX LM
With pip
pip install mlx-lmWith conda
conda install -c conda-forge mlx-lmRequirements
- Apple silicon Mac
- Python 3.11 or higher
- macOS 15.0 or higher for wiring memory to speed up large models
First steps
- Generate text with the default model: mlx_lm.generate --prompt "How tall is Mt Everest?"
- Start an interactive chat session with mlx_lm.chat.
- Pick a different model with the --model flag, such as any model from the MLX Community organization.
- Run mlx_lm.generate -h to list options for the model, sampling parameters, and more.
Who MLX LM is for
- Developers running LLMs locally on Apple silicon Macs
- Developers fine-tuning LLMs with LoRA or full fine-tuning on a Mac
- Developers quantizing Hugging Face models and publishing MLX versions
- Python developers who want a scriptable generation API with streaming and custom sampling
Limitations to know
- Models that are large relative to the machine's RAM can be slow, and the memory wiring speedup needs macOS 15 or higher.
- Small rotating key-value cache sizes save RAM but lower output quality.
- Some models need trust_remote_code enabled for their tokenizer.
Is MLX LM growing? Usage data
In the week of Oct 25–31, 2026, MLX LM ranked #320 on the Devtoolwatch Board and #39 in Inference and training, with a momentum score of 28 out of 100. Its main package had 137k downloads that week, −1% against its own 4-week average.
Show the weekly numbers
| Week of | New stars |
|---|---|
| Aug 9, 2026 | 89 |
| Aug 16, 2026 | 115 |
| Aug 23, 2026 | 91 |
| Aug 30, 2026 | 68 |
| Sep 6, 2026 | 90 |
| Sep 13, 2026 | 82 |
| Sep 20, 2026 | 85 |
| Sep 27, 2026 | 57 |
| Oct 4, 2026 | 58 |
| Oct 11, 2026 | 127 |
| Oct 18, 2026 | 84 |
| Oct 25, 2026 | 55 |
The Board ranks tools by four signals each week: new stars, new stars for the repo's size, added downloads and download growth beyond the market. How the ranking works.
MLX LM alternatives
Other LLM inference and training tools on the Board in the week of Oct 25–31, 2026, by momentum:
| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | dsh-our-free-modeldsh-our-free-model is a dsh plugin that provides DeepSeek V4.1 Flash, Kimi K3 and other models without login, sign-up or API key. | 54.3k | – |
| #2 | magpiemagpie is a local gateway and app that lets coding agents like Claude Code, Codex and Gemini CLI use any provider's model. | 26.9k | – |
| #3 | StrataStrata is a local inference engine and app that runs large open models like Qwen3.8-Flash-Next on consumer hardware with… | 37.1k | – |
| #4 | edge0edge0 is an open-source streaming inference framework that runs large sparse mixture-of-experts models on consumer hardware by streaming… | 8.9k | – |
| #5 | OmniRouteOmniRoute is an open-source AI gateway that routes requests across many model providers through one OpenAI-compatible endpoint, with… | 83.2k | 106k |
| #6 | CLIProxyAPICLIProxyAPI is a Go proxy server that exposes CLI coding accounts like Codex, Claude Code and Gemini CLI through OpenAI, Gemini and… | 57.1k | – |
Recent MLX LM releases
MLX LM FAQ
Is MLX LM free and open source?
Yes. MLX LM's source code is on GitHub (ml-explore/mlx-lm) under the MIT license, so you can use it for free.
How do I install MLX LM?
Run pip install mlx-lm. The mlx-lm package needs Python 3.11 or newer. The install section above has the other options from the project's README.
Is MLX LM still maintained?
Yes. The ml-explore/mlx-lm repository was last updated on Oct 20, 2026, and the latest release, v0.32.2 on GitHub, came out on Oct 16, 2026.
How popular is MLX LM?
MLX LM has 7.5k GitHub stars as of Nov 1, 2026. It gained 55 new stars in the week of Oct 25–31, 2026, against a 12-week average of 83 a week. Its main package had 137k downloads that week, −1% against its own 4-week average. That made it #39 among LLM inference and training tools on the Devtoolwatch Board.
MLX LM vs Ollama: how do they compare?
MLX LM and Ollama are both tracked by Devtoolwatch as LLM inference and training tools. MLX LM has 7.5k GitHub stars and Ollama has 185.5k. In the week of Oct 25–31, 2026, MLX LM's main package had 137k downloads and Ollama's had 4M. Ollama is a tool for running and building with open AI models locally or in the cloud on macOS, Windows, and Linux. Ollama review.
MLX LM vs vLLM: how do they compare?
MLX LM and vLLM are both tracked by Devtoolwatch as LLM inference and training tools. MLX LM has 7.5k GitHub stars and vLLM has 96k. In the week of Oct 25–31, 2026, MLX LM's main package had 137k downloads and vLLM's had 677.2k. vLLM is an open-source Python library and engine for high-throughput, memory-efficient inference and serving of large language models. vLLM review.
What are the best alternatives to MLX LM?
Other LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: dsh-our-free-model (#1), magpie (#2), Strata (#3), edge0 (#4), OmniRoute (#5). The alternatives table above has their numbers.
What is MLX LM?
MLX LM is a Python package from the ml-explore organization for generating text and fine-tuning large language models on Apple silicon with MLX. MLX LM integrates with the Hugging Face Hub and supports quantization, LoRA and full fine-tuning, and distributed inference.
Where does MLX LM store converted models?
MLX LM saves converted models to the path mlx_model by default when you use convert. MLX LM downloads source models from the Hugging Face Hub. The sources do not state the download cache location.
Which models can MLX LM run?
MLX LM runs any MLX-compatible model passed with the --model flag. Thousands are available in the MLX Community Hugging Face organization. The default model is mlx-community/Llama-3.2-3B-Instruct-4bit. If a model is not supported, you can file an issue or submit a pull request.
How do I speed up repeated queries on a long context in MLX LM?
MLX LM can cache a prompt with mlx_lm.cache_prompt and reuse it through --prompt-cache-file in mlx_lm.generate. The cached prompt is treated as a prefix, and the model is read from the cache. Prompt caching also works in the Python API.
Does MLX LM include a server?
MLX LM includes an mlx_lm.server command, which accepts options such as --prefill-step-size. Release notes mention fixes to tool call handling in the server. The README does not document server usage in detail.
Sources
MLX LM README badge
Maintainers can show the tool's weekly Board rank in their README:
[](https://index.devtoolwatch.com/tools/mlx-lm/)How this page is made: Devtoolwatch writes each review from the project's README, releases, package metadata and website, with an AI model, then checks every claim against the source it cites and drops what the sources don't support. The numbers come from Devtoolwatch's daily GitHub snapshots, GitHub's stargazer history, and the npm and PyPI download APIs. Spotted an error? Email [email protected].