# edge0 review

> edge0 is an open-source streaming inference framework that runs large sparse mixture-of-experts models on consumer hardware by streaming expert weights from SSD.

Devtoolwatch Board, the week of Oct 25–31, 2026. Canonical page: https://index.devtoolwatch.com/tools/edge0-ai-edge0/

## TL;DR

edge0 is an open-source streaming MoE inference framework that offloads expert weights to SSD, so peak memory depends on the active experts, not the parameter count. edge0 is for developers running large MoE models locally on Macs, phones and Windows PCs. A trained prerouter predicts expert routing one step ahead so weight loads overlap computation.

## edge0 at a glance

- **Category:** [Inference and training](https://index.devtoolwatch.com/categories/inference-and-training/)
- **Maintained by:** Edge0 (organization, [@Edge0-AI](https://github.com/Edge0-AI) on GitHub)
- **License:** Apache 2.0
- **Language:** Python, Rust, C++
- **Repository:** [github.com/Edge0-AI/Edge0](https://github.com/Edge0-AI/Edge0)
- **Latest release:** [v0.1.1](https://github.com/Edge0-AI/Edge0/releases/tag/v0.1.1) on GitHub, Oct 29, 2026
- **First published:** Sep 8, 2026

## What edge0 does

- edge0 streams expert weights from storage on demand, so peak memory is bounded by the active expert set.
- edge0 uses a trained prerouter head to predict expert routing one step ahead, overlapping expert loads with the forward pass.
- edge0 applies Recover-LoRA adapters, distilled from the FP teacher, to recover most of the int4 quantization loss.
- edge0 serves models through an OpenAI-compatible /v1/chat/completions endpoint.
- edge0 ships two model tiers, edge0-35b and edge0-8b, each bundled with trained LoRA and prerouter adapters.
- edge0 provides a transformers-style Python API with AutoModel, AutoConfig and AutoEngine.
- edge0 includes platform engines for macOS, iOS, Android and Windows.

### How it works

edge0's Python framework runs on MLX on Apple Silicon Macs. You download a model directory from Hugging Face or ModelScope that holds the base checkpoint and its adapters. Then you run the edge0 CLI to demo, chat or serve an OpenAI-compatible API. Expert weights are memory-mapped and read on demand. Separate native engines cover macOS (Rust), iOS (Swift), Android (Kotlin) and Windows (C++ with Vulkan).

## How to install edge0

**Python framework from source**

```
python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,fetch]'
```

### Requirements

- The Python framework requires macOS on Apple Silicon (M1/M2/M3/M4).
- Python 3.10 or later; 3.12 is recommended.
- MLX pinned to mlx==0.30.6, mlx-metal==0.30.6 and mlx-lm==0.31.0.
- About 23 GB of disk for edge0-35b or about 4.2 GB for edge0-8b.

### First steps

1. Download a tier with the helper script: .venv/bin/python scripts/fetch_models.py --tier edge0-8b --target-dir models
2. Point edge0 at the download with environment variables such as EDGE0_8B_MODEL, or pass the model directory directly.
3. Run a quick demo with edge0 demo edge0-35b.
4. Start an OpenAI-compatible server with edge0 serve edge0-35b and query /v1/chat/completions on port 8000.

## Who edge0 is for

- Developers running 35B-class MoE models on Apple Silicon Macs with limited RAM.
- Teams that want a local OpenAI-compatible chat endpoint backed by an on-device model.
- Mobile developers experimenting with on-device MoE inference on iPhone or Android.
- Researchers studying SSD expert offload and routing prediction for sparse MoE models.

### Limitations

- edge0's Python framework supports only macOS on Apple Silicon. The CUDA backend is still on the roadmap.
- edge0's int4 pipeline loses some quality versus the fp16 base models, according to the project's own benchmarks.
- edge0's unified cross-platform framework is not released yet. It is planned for Q4 2026, and the platform engines use separate stacks today.
- edge0's macOS app is ad-hoc signed and the Android APK is not on Google Play.

## Usage data (the week of Oct 25–31, 2026)

- Board rank: #8 overall, #4 in Inference and training, momentum score 96
- New GitHub stars per week (last 12 weeks): Aug 9: n/a; Aug 16: n/a; Aug 23: n/a; Aug 30: n/a; Sep 6: 1.5k; Sep 13: 529; Sep 20: 95; Sep 27: 650; Oct 4: 1.3k; Oct 11: 1.1k; Oct 18: 1.5k; Oct 25: 2.3k

## edge0 alternatives

| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | [dsh-our-free-model](https://index.devtoolwatch.com/tools/ebony-vinyl-dsh-our-free-model/) | 54.3k | – |
| #2 | [magpie](https://index.devtoolwatch.com/tools/yetone-magpie/) | 26.9k | – |
| #3 | [Strata](https://index.devtoolwatch.com/tools/niko1221-strata/) | 37.1k | – |
| #5 | [OmniRoute](https://index.devtoolwatch.com/tools/diegosouzapw-omniroute/) | 83.2k | 106k |
| #6 | [CLIProxyAPI](https://index.devtoolwatch.com/tools/router-for-me-cliproxyapi/) | 57.1k | – |
| #7 | [codex-chatgpt-web](https://index.devtoolwatch.com/tools/miuuyy-codex-chatgpt-web/) | 15.3k | – |

## edge0 FAQ

### Is edge0 free and open source?

Yes. edge0's source code is on GitHub (Edge0-AI/Edge0) under the Apache 2.0 license, so you can use it for free.

### How do I install edge0?

Run `python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,fetch]'`. The install section above has the other options from the project's README.

### Is edge0 still maintained?

Yes. The Edge0-AI/Edge0 repository was last updated on Oct 29, 2026, and the latest release, v0.1.1 on GitHub, came out on Oct 29, 2026.

### How popular is edge0?

edge0 has 8.9k GitHub stars as of Nov 1, 2026. It gained 2.3k new stars in the week of Oct 25–31, 2026, against a 12-week average of 1.1k a week. That made it #4 among LLM inference and training tools on the Devtoolwatch Board.

### What are the best alternatives to edge0?

Other LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: dsh-our-free-model (#1), magpie (#2), Strata (#3), OmniRoute (#5), CLIProxyAPI (#6). The alternatives table above has their numbers.

### How much memory does edge0 need?

edge0 needs about 2.9 GB of MLX allocator peak memory for edge0-35b and about 1.0 GB for edge0-8b at short contexts. Process RSS is higher but reclaimable, and the same build runs on a 16 GB M2. Leave headroom for the OS and long-context KV growth.

### Does edge0 offer an OpenAI-compatible API?

Yes. The command edge0 serve exposes an OpenAI-compatible /v1/chat/completions endpoint, by default at http://127.0.0.1:8000.

### Can edge0 run a model without the LoRA or prerouter adapters?

Yes. Pass --no-prerouter or --no-lora to run the plain base model. Without those flags, edge0 requires both adapter files and fails with a clear message if one is missing.

### Why does edge0 produce garbled output on Apple A18 chips?

Garbled, mixed-language output on Apple A18 or A18 Pro means an older mlx version is installed. The fix is to install mlx==0.30.6 and mlx-metal==0.30.6.

### Which base models are the edge0 tiers built on?

edge0-35b is built on Qwen3.6-35B-A3B, and edge0-8b is built on the Ling 3.0 bailing hybrid. Both tiers are 4-bit and ship with trained LoRA and prerouter adapters.

### How do I open the edge0 macOS app if Gatekeeper blocks it?

The edge0 macOS app is ad-hoc signed, so Gatekeeper blocks it after download. Clear the quarantine attribute with xattr -dr com.apple.quarantine edge0-0.1.0-arm64.app, then open the app.

## Sources

1. [Edge0-AI/Edge0 on GitHub](https://github.com/Edge0-AI/Edge0)
2. [edge0 README](https://github.com/Edge0-AI/Edge0/blob/main/README.md)
3. [Release v0.1.1 (Oct 29, 2026)](https://github.com/Edge0-AI/Edge0/releases/tag/v0.1.1)
4. [Release v0.1.0 (Oct 21, 2026)](https://github.com/Edge0-AI/Edge0/releases/tag/v0.1.0)
5. [Devtoolwatch Board, the week of Oct 25–31, 2026](https://index.devtoolwatch.com/)