Devtoolwatch Boardedge0 review
edge0 is an open-source streaming inference framework that runs large sparse mixture-of-experts models on consumer hardware by streaming expert weights from SSD.
TL;DR
edge0 is an open-source streaming MoE inference framework that offloads expert weights to SSD, so peak memory depends on the active experts, not the parameter count. edge0 is for developers running large MoE models locally on Macs, phones and Windows PCs. A trained prerouter predicts expert routing one step ahead so weight loads overlap computation.
- 8.9kGitHub stars
- +2.3knew stars, week of Oct 25
- #4in Inference and training this week
edge0 at a glance
- Category
- Inference and training
- Maintained by
- Edge0 (organization, @Edge0-AI on GitHub)
- License
- Apache 2.0
- Language
- Python, Rust, C++
- Repository
- github.com/Edge0-AI/Edge0
- Latest release
- v0.1.1 on GitHub, Oct 29, 2026
- First published
- Sep 8, 2026
What edge0 does
- edge0 streams expert weights from storage on demand, so peak memory is bounded by the active expert set.
- edge0 uses a trained prerouter head to predict expert routing one step ahead, overlapping expert loads with the forward pass.
- edge0 applies Recover-LoRA adapters, distilled from the FP teacher, to recover most of the int4 quantization loss.
- edge0 serves models through an OpenAI-compatible /v1/chat/completions endpoint.
- edge0 ships two model tiers, edge0-35b and edge0-8b, each bundled with trained LoRA and prerouter adapters.
- edge0 provides a transformers-style Python API with AutoModel, AutoConfig and AutoEngine.
- edge0 includes platform engines for macOS, iOS, Android and Windows.
How it works
edge0's Python framework runs on MLX on Apple Silicon Macs. You download a model directory from Hugging Face or ModelScope that holds the base checkpoint and its adapters. Then you run the edge0 CLI to demo, chat or serve an OpenAI-compatible API. Expert weights are memory-mapped and read on demand. Separate native engines cover macOS (Rust), iOS (Swift), Android (Kotlin) and Windows (C++ with Vulkan).
How to install and start using edge0
Python framework from source
python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,fetch]'Requirements
- The Python framework requires macOS on Apple Silicon (M1/M2/M3/M4).
- Python 3.10 or later; 3.12 is recommended.
- MLX pinned to mlx==0.30.6, mlx-metal==0.30.6 and mlx-lm==0.31.0.
- About 23 GB of disk for edge0-35b or about 4.2 GB for edge0-8b.
First steps
- Download a tier with the helper script: .venv/bin/python scripts/fetch_models.py --tier edge0-8b --target-dir models
- Point edge0 at the download with environment variables such as EDGE0_8B_MODEL, or pass the model directory directly.
- Run a quick demo with edge0 demo edge0-35b.
- Start an OpenAI-compatible server with edge0 serve edge0-35b and query /v1/chat/completions on port 8000.
Who edge0 is for
- Developers running 35B-class MoE models on Apple Silicon Macs with limited RAM.
- Teams that want a local OpenAI-compatible chat endpoint backed by an on-device model.
- Mobile developers experimenting with on-device MoE inference on iPhone or Android.
- Researchers studying SSD expert offload and routing prediction for sparse MoE models.
Limitations to know
- edge0's Python framework supports only macOS on Apple Silicon. The CUDA backend is still on the roadmap.
- edge0's int4 pipeline loses some quality versus the fp16 base models, according to the project's own benchmarks.
- edge0's unified cross-platform framework is not released yet. It is planned for Q4 2026, and the platform engines use separate stacks today.
- edge0's macOS app is ad-hoc signed and the Android APK is not on Google Play.
Is edge0 growing? Usage data
In the week of Oct 25–31, 2026, edge0 ranked #8 on the Devtoolwatch Board and #4 in Inference and training, with a momentum score of 96 out of 100.
Show the weekly numbers
| Week of | New stars |
|---|---|
| Aug 9, 2026 | – |
| Aug 16, 2026 | – |
| Aug 23, 2026 | – |
| Aug 30, 2026 | – |
| Sep 6, 2026 | 1.5k |
| Sep 13, 2026 | 529 |
| Sep 20, 2026 | 95 |
| Sep 27, 2026 | 650 |
| Oct 4, 2026 | 1.3k |
| Oct 11, 2026 | 1.1k |
| Oct 18, 2026 | 1.5k |
| Oct 25, 2026 | 2.3k |
The Board ranks tools by four signals each week: new stars, new stars for the repo's size, added downloads and download growth beyond the market. How the ranking works.
edge0 alternatives
Other LLM inference and training tools on the Board in the week of Oct 25–31, 2026, by momentum:
| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | dsh-our-free-modeldsh-our-free-model is a dsh plugin that provides DeepSeek V4.1 Flash, Kimi K3 and other models without login, sign-up or API key. | 54.3k | – |
| #2 | magpiemagpie is a local gateway and app that lets coding agents like Claude Code, Codex and Gemini CLI use any provider's model. | 26.9k | – |
| #3 | StrataStrata is a local inference engine and app that runs large open models like Qwen3.8-Flash-Next on consumer hardware with… | 37.1k | – |
| #5 | OmniRouteOmniRoute is an open-source AI gateway that routes requests across many model providers through one OpenAI-compatible endpoint, with… | 83.2k | 106k |
| #6 | CLIProxyAPICLIProxyAPI is a Go proxy server that exposes CLI coding accounts like Codex, Claude Code and Gemini CLI through OpenAI, Gemini and… | 57.1k | – |
| #7 | codex-chatgpt-webcodex-chatgpt-web is a launcher that lets Codex use the ChatGPT Web models on your account, including Pro, through browser automation. | 15.3k | – |
Recent edge0 releases
edge0 FAQ
Is edge0 free and open source?
Yes. edge0's source code is on GitHub (Edge0-AI/Edge0) under the Apache 2.0 license, so you can use it for free.
How do I install edge0?
Run python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,fetch]'. The install section above has the other options from the project's README.
Is edge0 still maintained?
Yes. The Edge0-AI/Edge0 repository was last updated on Oct 29, 2026, and the latest release, v0.1.1 on GitHub, came out on Oct 29, 2026.
How popular is edge0?
edge0 has 8.9k GitHub stars as of Nov 1, 2026. It gained 2.3k new stars in the week of Oct 25–31, 2026, against a 12-week average of 1.1k a week. That made it #4 among LLM inference and training tools on the Devtoolwatch Board.
What are the best alternatives to edge0?
Other LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: dsh-our-free-model (#1), magpie (#2), Strata (#3), OmniRoute (#5), CLIProxyAPI (#6). The alternatives table above has their numbers.
How much memory does edge0 need?
edge0 needs about 2.9 GB of MLX allocator peak memory for edge0-35b and about 1.0 GB for edge0-8b at short contexts. Process RSS is higher but reclaimable, and the same build runs on a 16 GB M2. Leave headroom for the OS and long-context KV growth.
Does edge0 offer an OpenAI-compatible API?
Yes. The command edge0 serve exposes an OpenAI-compatible /v1/chat/completions endpoint, by default at http://127.0.0.1:8000.
Can edge0 run a model without the LoRA or prerouter adapters?
Yes. Pass --no-prerouter or --no-lora to run the plain base model. Without those flags, edge0 requires both adapter files and fails with a clear message if one is missing.
Why does edge0 produce garbled output on Apple A18 chips?
Garbled, mixed-language output on Apple A18 or A18 Pro means an older mlx version is installed. The fix is to install mlx==0.30.6 and mlx-metal==0.30.6.
Which base models are the edge0 tiers built on?
edge0-35b is built on Qwen3.6-35B-A3B, and edge0-8b is built on the Ling 3.0 bailing hybrid. Both tiers are 4-bit and ship with trained LoRA and prerouter adapters.
How do I open the edge0 macOS app if Gatekeeper blocks it?
The edge0 macOS app is ad-hoc signed, so Gatekeeper blocks it after download. Clear the quarantine attribute with xattr -dr com.apple.quarantine edge0-0.1.0-arm64.app, then open the app.
Sources
edge0 README badge
Maintainers can show the tool's weekly Board rank in their README:
[](https://index.devtoolwatch.com/tools/edge0-ai-edge0/)How this page is made: Devtoolwatch writes each review from the project's README, releases, package metadata and website, with an AI model, then checks every claim against the source it cites and drops what the sources don't support. The numbers come from Devtoolwatch's daily GitHub snapshots, GitHub's stargazer history, and the npm and PyPI download APIs. Spotted an error? Email [email protected].