Devtoolwatch BoardBest LLM inference and training tools, ranked by real usage
45 LLM inference and training tools ranked by new GitHub stars and npm and PyPI download growth, the week of Oct 25–31, 2026. Updated every Monday.
| Rank | Tool | Stars | Downloads/wk | vs 4-wk avg |
|---|---|---|---|---|
| #1 | dsh-our-free-modeldsh-our-free-model is a dsh plugin that provides DeepSeek V4.1 Flash, Kimi K3 and other models without login, sign-up or API key. | 54.3k | – | – |
| #2 | magpiemagpie is a local gateway and app that lets coding agents like Claude Code, Codex and Gemini CLI use any provider's model. | 26.9k | – | – |
| #3 | StrataStrata is a local inference engine and app that runs large open models like Qwen3.8-Flash-Next on consumer hardware with… | 37.1k | – | – |
| #4 | edge0edge0 is an open-source streaming inference framework that runs large sparse mixture-of-experts models on consumer hardware by streaming… | 8.9k | – | – |
| #5 | OmniRouteOmniRoute is an open-source AI gateway that routes requests across many model providers through one OpenAI-compatible endpoint, with… | 83.2k | 106k | +55% |
| #6 | CLIProxyAPICLIProxyAPI is a Go proxy server that exposes CLI coding accounts like Codex, Claude Code and Gemini CLI through OpenAI, Gemini and… | 57.1k | – | – |
| #7 | codex-chatgpt-webcodex-chatgpt-web is a launcher that lets Codex use the ChatGPT Web models on your account, including Pro, through browser automation. | 15.3k | – | – |
| #8 | 9Router9Router is a local AI router that connects coding tools like Claude Code, Codex, Cursor and Cline to many AI providers through one… | 31.8k | 108.8k | +84% |
| #9 | vLLMvLLM is an open-source Python library and engine for high-throughput, memory-efficient inference and serving of large language models. | 96k | 677.2k | +43% |
| #10 | audio.cppAn all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation,… | 3.9k | – | – |
| #11 | Sub2APISub2API 一站式开源中转服务，让 Claude、Openai 、Gemini、Grok订阅统一接入，支持拼车共享，更高效分摊成本，原生工具无缝使用。 | 45.2k | – | – |
| #12 | llama.cppllama.cpp is a plain C/C++ library and set of tools for running LLM and VLM inference locally or in the cloud on many kinds of hardware. | 133k | – | – |
| #13 | LibreChatLibreChat is an open-source, self-hosted AI chat platform that unifies many AI providers with agents, MCP, artifacts, and multi-user… | 46.6k | – | – |
| #14 | TransformersTransformers is a Python model-definition framework for text, vision, audio, video, and multimodal machine learning models, used for… | 168.4k | 27.8M | +11% |
| #15 | FlashInferFlashInfer is a GPU kernel library and kernel generator for LLM inference, covering attention, GEMM, MoE, sampling and communication… | 6.8k | 938.6k | +18% |
| #16 | TurboFieldfareGemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook | 7.1k | – | – |
| #17 | AnythingLLMAnythingLLM is a self-hostable, local-first AI app for chatting with your documents, running AI agents, and connecting local or cloud LLMs. | 67.8k | – | – |
| #18 | kimi-k3-in-cA 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU. | 9.3k | – | – |
| #19 | DynamoDynamo is an open-source, datacenter-scale distributed inference serving framework that coordinates SGLang, TensorRT-LLM, and vLLM… | 8.4k | 20.7k | +60% |
| #20 | LocalAILocalAI is an open-source AI engine that runs LLMs, vision, voice, image and video models on any hardware, with no GPU required. | 50.1k | – | – |
| #21 | MLXMLX is an array framework for machine learning on Apple silicon, from Apple machine learning research, with a NumPy-like Python API. | 29k | 356.9k | +13% |
| #22 | LayaNon-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+… | 34.8k | 54.5k | −28% |
| #23 | SGLangSGLang is an open-source inference and serving framework for large language, vision-language, and diffusion models, with… | 38.3k | 336.7k | −18% |
| #24 | Rapid-MLXRapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused… | 4.1k | 2.1k | +16% |
| #25 | LMDeployLMDeploy is a toolkit for compressing, deploying, and serving LLMs. | 8.1k | 13.7k | +145% |
| #26 | GPUStackA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances. | 5.9k | – | – |
| #27 | JanJan is an open source alternative to ChatGPT that runs 100% offline on your computer. | 45.2k | – | – |
| #28 | LemonadeLemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord:… | 6k | – | – |
| #29 | OllamaOllama is a tool for running and building with open AI models locally or in the cloud on macOS, Windows, and Linux. | 185.5k | 4M | −2% |
| #30 | BentoMLBentoML is a Python library for building online serving systems and inference APIs for AI apps and models. | 8.9k | 51.1k | +29% |
| #31 | Open WebUIUser-friendly AI Interface (Supports Ollama, OpenAI API, ...) | 157.4k | 158.6k | −17% |
| #32 | UnslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more. | 78.9k | 170.5k | −7% |
| #33 | MTPLXThe fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a… | 2.7k | 443 | −45% |
| #34 | PEFTPEFT is a Hugging Face Python library for adapting large pretrained models by fine-tuning only a small number of extra parameters. | 21.9k | 2.2M | +6% |
| #35 | OpenVINOOpenVINO™ is an open source toolkit for optimizing and deploying AI inference | 11.1k | 228.8k | +3% |
| #36 | exoRun frontier AI locally. | 48.1k | – | – |
| #37 | KServeStandardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes | 6.2k | 13k | +8% |
| #38 | verlverl/HybridFlow: A Flexible and Efficient RL Post-Training Framework | 24.1k | 4.2k | −3% |
| #39 | MLX LMMLX LM is a Python package for generating text with and fine-tuning large language models on Apple silicon using MLX. | 7.5k | 137k | −1% |
| #40 | LLaMA-FactoryUnified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024) | 75.8k | 3.3k | −24% |
| #41 | Laya-MLXNative MLX runtime for Laya typed decision models — 7–14 ms short decisions on M3 Max. No text generation, PyTorch, or cloud API. | 7.1k | 724 | −60% |
| #42 | GPT4AllGPT4All is an open-source desktop app and Python client for running large language models locally on everyday desktops and laptops. | 77.3k | 11.9k | +11% |
| #43 | TensorRT-LLMTensorRT-LLM is an NVIDIA library with a Python API for defining large language models and running optimized inference on NVIDIA GPUs. | 15k | 1.3k | −42% |
| #44 | llamafileDistribute and run LLMs with a single file. | 26.5k | – | – |
| #45 | TRLTrain transformer language models with reinforcement learning. | 19.6k | 536k | −2% |
FAQ
What is the fastest-growing of the LLM inference and training tools right now?
dsh-our-free-model ranked #1 among LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, with a momentum score of 100 out of 100.
Which of the LLM inference and training tools has the most GitHub stars?
Of the LLM inference and training tools ranked in the week of Oct 25–31, 2026, Ollama has the most GitHub stars: 185.5k.
Which of the LLM inference and training tools is downloaded the most?
Transformers's main package had the most downloads of the ranked LLM inference and training tools: 27.8M in the week of Oct 25–31, 2026.
How does Devtoolwatch rank LLM inference and training tools?
Every Monday the Board averages four percentile ranks: new GitHub stars from daily snapshots, new stars for the repo's size, added npm and PyPI downloads, and download growth beyond the market. Star spikes without forks, downloads or Hacker News to back them are held back.
Rankings come from Devtoolwatch's daily GitHub snapshots and the npm and PyPI download APIs. How the ranking works.