vLLM review

Q: How do I install vLLM?

Run uv pip install vllm. The vllm package needs Python 3.11 or newer, below 3.15. The install section above has the other options from the project's README.

vLLM is an open-source Python library and engine for high-throughput, memory-efficient inference and serving of large language models.

Updated Nov 1, 2026. Review written Oct 25, 2026 from the project's own sources; usage data refreshed every Monday.

TL;DR

vLLM is a library for LLM inference and serving that began at UC Berkeley's Sky Computing Lab. vLLM is for teams serving Hugging Face models on GPUs, CPUs, or other accelerators. What sets vLLM apart is PagedAttention memory management, continuous batching, and an OpenAI-compatible API server.

96kGitHub stars
+668new stars, week of Oct 25
677.2kdownloads a week, +43% vs 4-week avg
#9in Inference and training this week

vLLM at a glance

Category: Inference and training
Maintained by: vLLM (organization, @vllm-project on GitHub)
License: Apache 2.0
Language: Python, Rust, Cuda
Repository: github.com/vllm-project/vllm
Website: vllm.ai
PyPI package: vllm 0.32.0
Latest release: v0.32.0 on GitHub, Oct 18, 2026
First published: Feb 9, 2023

What vLLM does

vLLM manages attention key and value memory with PagedAttention and uses continuous batching, chunked prefill, and prefix caching for incoming requests.
vLLM runs an OpenAI-compatible API server and also supports the Anthropic Messages API and gRPC.
vLLM supports quantization formats including FP8, INT8, INT4, GPTQ/AWQ, GGUF, and compressed-tensors.
vLLM offers tensor, pipeline, data, expert, and context parallelism for distributed inference.
vLLM provides speculative decoding methods, including n-gram, suffix, and EAGLE.
vLLM generates structured outputs using xgrammar or guidance and includes tool calling and reasoning parsers.
vLLM supports efficient multi-LoRA serving for dense and MoE layers, plus streaming outputs.

How it works

vLLM installs as a Python package and loads models from Hugging Face. These include decoder-only, mixture-of-experts, multimodal, embedding, and reward models. vLLM serves them through an OpenAI-compatible API server. vLLM runs on NVIDIA, AMD, and Intel GPUs and on x86, ARM, and PowerPC CPUs. Hardware plugins add Google TPUs, Intel Gaudi, Huawei Ascend, Apple Silicon, and others.

How to install and start using vLLM

With uv

uv pip install vllm

With uv, automatic torch backend

uv pip install vllm --torch-backend auto

Download precompiled kernels

vllm download-kernels

Requirements

vLLM 0.32.0 requires Python 3.11 or newer and below 3.15.
vLLM needs a supported accelerator or CPU, such as an NVIDIA, AMD, or Intel GPU or an x86/ARM/PowerPC CPU.

First steps

Install vLLM with uv, which the project recommends over pip.
Run vllm download-kernels to fetch precompiled kernels for faster startup.
Follow the Quickstart guide at docs.vllm.ai and check the supported models list.

Who vLLM is for

Teams serving open-source LLMs behind an OpenAI-compatible API.
Developers running large mixture-of-experts models across multiple GPUs with distributed parallelism.
Researchers studying LLM serving memory management and scheduling.
Operators deploying on mixed hardware, including AMD ROCm, TPUs, or Intel Gaudi.

Limitations to know

vLLM supports some hardware, such as TPUs, Gaudi, and Apple Silicon, through separate plugins rather than the core package.
vLLM's Python requirements differ between sources: PyPI states 3.11 to below 3.15, while the homepage says 3.10+.

Is vLLM growing? Usage data

In the week of Oct 25–31, 2026, vLLM ranked #56 on the Devtoolwatch Board and #9 in Inference and training, with a momentum score of 74 out of 100. Its main package had 677.2k downloads that week, +43% against its own 4-week average.

New GitHub stars per week for the last 12 weeks, from GitHub's stargazer history (today's stargazers by the week they starred).

Show the weekly numbers

New GitHub stars per week for vLLM
Week of	New stars
Aug 9, 2026	627
Aug 16, 2026	639
Aug 23, 2026	738
Aug 30, 2026	627
Sep 6, 2026	589
Sep 13, 2026	628
Sep 20, 2026	558
Sep 27, 2026	425
Oct 4, 2026	449
Oct 11, 2026	525
Oct 18, 2026	1.4k
Oct 25, 2026	668

On Hacker News

vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention, 295 points and 42 comments (Jun 20, 2023)
vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep, 147 points and 54 comments (Jan 13, 2026)
Speculative Decoding in vLLM on AMD GPUs, 145 points and 53 comments (Sep 7, 2026)
vLLM v0.28.0, 108 points and 43 comments (Aug 29, 2026)
Micro-Agent: Beat Frontier Models with Collaboration Inside Model API, 81 points and 22 comments (Jun 29, 2026)

The Board ranks tools by four signals each week: new stars, new stars for the repo's size, added downloads and download growth beyond the market. How the ranking works.

vLLM alternatives

Other LLM inference and training tools on the Board in the week of Oct 25–31, 2026, by momentum:

Inference and training ranked by momentum, the week of Oct 25–31, 2026
Rank	Tool	Stars	Downloads/wk
#1	dsh-our-free-modeldsh-our-free-model is a dsh plugin that provides DeepSeek V4.1 Flash, Kimi K3 and other models without login, sign-up or API key.	54.3k	–
#2	magpiemagpie is a local gateway and app that lets coding agents like Claude Code, Codex and Gemini CLI use any provider's model.	26.9k	–
#3	StrataStrata is a local inference engine and app that runs large open models like Qwen3.8-Flash-Next on consumer hardware with…	37.1k	–
#4	edge0edge0 is an open-source streaming inference framework that runs large sparse mixture-of-experts models on consumer hardware by streaming…	8.9k	–
#5	OmniRouteOmniRoute is an open-source AI gateway that routes requests across many model providers through one OpenAI-compatible endpoint, with…	83.2k	106k
#6	CLIProxyAPICLIProxyAPI is a Go proxy server that exposes CLI coding accounts like Codex, Claude Code and Gemini CLI through OpenAI, Gemini and…	57.1k	–

All LLM inference and training tools, ranked

Recent vLLM releases

v0.32.0, Oct 18, 2026
v0.31.0, Oct 5, 2026
v0.30.0, Sep 22, 2026
v0.29.0, Sep 9, 2026
v0.28.0, Aug 26, 2026

vLLM FAQ

Is vLLM free and open source?

Yes. vLLM's source code is on GitHub (vllm-project/vllm) under the Apache 2.0 license, so you can use it for free.

How do I install vLLM?

Run uv pip install vllm. The vllm package needs Python 3.11 or newer, below 3.15. The install section above has the other options from the project's README.

Is vLLM still maintained?

Yes. The vllm-project/vllm repository was last updated on Oct 25, 2026, and the latest release, v0.32.0 on GitHub, came out on Oct 18, 2026.

How popular is vLLM?

vLLM has 96k GitHub stars as of Nov 1, 2026. It gained 668 new stars in the week of Oct 25–31, 2026, against a 12-week average of 658 a week. Its main package had 677.2k downloads that week, +43% against its own 4-week average. That made it #9 among LLM inference and training tools on the Devtoolwatch Board.

vLLM vs llama.cpp: how do they compare?

vLLM and llama.cpp are both tracked by Devtoolwatch as LLM inference and training tools. vLLM has 96k GitHub stars and llama.cpp has 133k. llama.cpp is a plain C/C++ library and set of tools for running LLM and VLM inference locally or in the cloud on many kinds of hardware. llama.cpp review.

vLLM vs Ollama: how do they compare?

vLLM and Ollama are both tracked by Devtoolwatch as LLM inference and training tools. vLLM has 96k GitHub stars and Ollama has 185.5k. In the week of Oct 25–31, 2026, vLLM's main package had 677.2k downloads and Ollama's had 4M. Ollama is a tool for running and building with open AI models locally or in the cloud on macOS, Windows, and Linux. Ollama review.

What are the best alternatives to vLLM?

Other LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: dsh-our-free-model (#1), magpie (#2), Strata (#3), edge0 (#4), OmniRoute (#5). The alternatives table above has their numbers.

Is vLLM an inference engine?

Yes. vLLM describes itself as a high-throughput and memory-efficient inference and serving engine for LLMs.

Is vLLM written in Python?

Yes. vLLM is mainly Python. The repository also contains Rust, CUDA, C++, and Shell code.

How do I use vLLM with Hugging Face models?

vLLM integrates with Hugging Face models. It supports decoder-only, mixture-of-experts, multimodal, embedding, and reward models. The full list is in the documentation.

Can vLLM run on a Mac?

vLLM lists Apple Silicon among its hardware plugins. Apple Silicon support comes from a plugin rather than core GPU support.

Does vLLM provide an OpenAI-compatible API?

Yes. vLLM includes an OpenAI-compatible API server. It also supports the Anthropic Messages API and gRPC.

Sources

vLLM README badge

Maintainers can show the tool's weekly Board rank in their README:

[![Devtoolwatch](https://index.devtoolwatch.com/badge/vllm.svg)](https://index.devtoolwatch.com/tools/vllm/)

How this page is made: Devtoolwatch writes each review from the project's README, releases, package metadata and website, with an AI model, then checks every claim against the source it cites and drops what the sources don't support. The numbers come from Devtoolwatch's daily GitHub snapshots, GitHub's stargazer history, and the npm and PyPI download APIs. Spotted an error? Email [email protected].