BentoML review

Q: How do I install BentoML?

Run pip install -U bentoml. The bentoml package needs Python 3.9 or newer. The install section above has the other options from the project's README.

BentoML is a Python library for building online serving systems and inference APIs for AI apps and models.

Updated Nov 1, 2026. Review written Oct 22, 2026 from the project's own sources; usage data refreshed every Monday.

TL;DR

BentoML is a Python library that turns model inference scripts into REST API servers using standard Python type hints. BentoML is for developers serving open-source or custom models in production. It packages code, models and dependencies into a deployable artifact called a Bento and generates Docker images automatically.

8.9kGitHub stars
+12new stars, week of Oct 25
51.1kdownloads a week, +29% vs 4-week avg
#30in Inference and training this week

BentoML at a glance

Category: Inference and training
Maintained by: BentoML (organization, @bentoml on GitHub)
License: Apache 2.0
Language: Python, Shell, Jinja
Repository: github.com/bentoml/BentoML
Website: bentoml.com
PyPI package: bentoml 1.4.39
Latest release: v1.4.39 on GitHub, May 7, 2026
First published: Apr 2, 2019

What BentoML does

BentoML turns a model inference script into a REST API server with a few lines of code and standard Python type hints.
BentoML manages environments, dependencies and model versions through a config file and automatically generates Docker images.
BentoML includes serving optimizations such as dynamic batching, model parallelism, multi-stage pipelines and multi-model inference-graph orchestration.
BentoML lets you implement custom APIs or task queues with your own business logic and multi-model composition.
BentoML supports any ML framework, modality and inference runtime.
BentoML deploys to production with Docker containers or Kubernetes after local development and debugging.

How it works

You define a service class in a service.py file with the @bentoml.service and @bentoml.api decorators. Running bentoml serve starts an HTTP server on localhost:3000. You can call it from a browser or from a Python client. bentoml build packages the code, models and dependency configs into a Bento. bentoml containerize turns the Bento into a Docker image.

How to install and start using BentoML

With pip

pip install -U bentoml

Requirements

Python 3.9 or later
Docker must be running to generate container images

First steps

Define your APIs in a service.py file using the @bentoml.service and @bentoml.api decorators.
Install your model's dependencies, such as torch and transformers, into your Python virtual environment for local runs.
Run bentoml serve to serve locally at http://localhost:3000, then call it with bentoml.SyncHTTPClient.
Run bentoml build, then bentoml containerize to produce a Docker image for deployment.

Who BentoML is for

Python developers turning model inference scripts into REST APIs
Teams building multi-model pipelines or LLM apps with batching and parallelism
Developers who want reproducible Docker images for deploying models to Kubernetes

Limitations to know

BentoML is a Python library, so services are written in Python.
BentoML collects anonymous usage data by default. You can opt out with --do-not-track or BENTOML_DO_NOT_TRACK=True.
BentoML's containerization step requires a running Docker installation.

Is BentoML growing? Usage data

In the week of Oct 25–31, 2026, BentoML ranked #274 on the Devtoolwatch Board and #30 in Inference and training, with a momentum score of 35 out of 100. Its main package had 51.1k downloads that week, +29% against its own 4-week average.

New GitHub stars per week for the last 12 weeks, from GitHub's stargazer history (today's stargazers by the week they starred).

Show the weekly numbers

New GitHub stars per week for BentoML
Week of	New stars
Aug 9, 2026	19
Aug 16, 2026	11
Aug 23, 2026	13
Aug 30, 2026	11
Sep 6, 2026	19
Sep 13, 2026	17
Sep 20, 2026	13
Sep 27, 2026	13
Oct 4, 2026	20
Oct 11, 2026	12
Oct 18, 2026	15
Oct 25, 2026	12

On Hacker News

LLM Inference Handbook, 366 points and 26 comments (Jul 11, 2025)
Navigating the World of Large Language Models, 48 points and 4 comments (Mar 22, 2024)
Show HN: BentoML goes 1.0 – A faster way to ship your models to production, 35 points and 13 comments (Jul 13, 2022)
Is LMDeploy the Ultimate Solution? Why It Outshines VLLM, TRT-LLM, TGI, and MLC, 16 points and 8 comments (Jun 20, 2024)
Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TensorRT-LLM, TGI, 15 points and 1 comments (Jul 5, 2024)

The Board ranks tools by four signals each week: new stars, new stars for the repo's size, added downloads and download growth beyond the market. How the ranking works.

BentoML alternatives

Other LLM inference and training tools on the Board in the week of Oct 25–31, 2026, by momentum:

Inference and training ranked by momentum, the week of Oct 25–31, 2026
Rank	Tool	Stars	Downloads/wk
#1	dsh-our-free-modeldsh-our-free-model is a dsh plugin that provides DeepSeek V4.1 Flash, Kimi K3 and other models without login, sign-up or API key.	54.3k	–
#2	magpiemagpie is a local gateway and app that lets coding agents like Claude Code, Codex and Gemini CLI use any provider's model.	26.9k	–
#3	StrataStrata is a local inference engine and app that runs large open models like Qwen3.8-Flash-Next on consumer hardware with…	37.1k	–
#4	edge0edge0 is an open-source streaming inference framework that runs large sparse mixture-of-experts models on consumer hardware by streaming…	8.9k	–
#5	OmniRouteOmniRoute is an open-source AI gateway that routes requests across many model providers through one OpenAI-compatible endpoint, with…	83.2k	106k
#6	CLIProxyAPICLIProxyAPI is a Go proxy server that exposes CLI coding accounts like Codex, Claude Code and Gemini CLI through OpenAI, Gemini and…	57.1k	–

All LLM inference and training tools, ranked

Recent BentoML releases

v1.4.39, May 7, 2026
v1.4.38, Apr 2, 2026
v1.4.37, Mar 25, 2026
v1.4.36, Mar 6, 2026
v1.4.35, Feb 3, 2026

BentoML FAQ

Is BentoML free and open source?

Yes. BentoML's source code is on GitHub (bentoml/BentoML) under the Apache 2.0 license, so you can use it for free. Its website also links to a pricing page (https://www.modular.com/pricing?utm_source=bentoml_pricing) for any paid plans.

How do I install BentoML?

Run pip install -U bentoml. The bentoml package needs Python 3.9 or newer. The install section above has the other options from the project's README.

Is BentoML still maintained?

Yes. The bentoml/BentoML repository was last updated on Oct 20, 2026, and the latest release, v1.4.39 on GitHub, came out on May 7, 2026.

How popular is BentoML?

BentoML has 8.9k GitHub stars as of Nov 1, 2026. It gained 12 new stars in the week of Oct 25–31, 2026, against a 12-week average of 15 a week. Its main package had 51.1k downloads that week, +29% against its own 4-week average. That made it #30 among LLM inference and training tools on the Devtoolwatch Board.

BentoML vs MLflow: how do they compare?

BentoML and MLflow are both tracked by Devtoolwatch. BentoML has 8.9k GitHub stars and MLflow has 28.6k. In the week of Oct 25–31, 2026, BentoML's main package had 51.1k downloads and MLflow's had 5.2M. MLflow is an open source AI engineering platform for tracing, evaluating, monitoring, and managing agents, LLM applications, and ML models. MLflow review.

BentoML vs vLLM: how do they compare?

BentoML and vLLM are both tracked by Devtoolwatch as LLM inference and training tools. BentoML has 8.9k GitHub stars and vLLM has 96k. In the week of Oct 25–31, 2026, BentoML's main package had 51.1k downloads and vLLM's had 677.2k. vLLM is an open-source Python library and engine for high-throughput, memory-efficient inference and serving of large language models. vLLM review.

What are the best alternatives to BentoML?

Other LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: dsh-our-free-model (#1), magpie (#2), Strata (#3), edge0 (#4), OmniRoute (#5). The alternatives table above has their numbers.

What is BentoML used for?

BentoML is used to build model inference APIs and multi-model serving systems. The project lists examples for LLMs, image generation, embeddings, audio and computer vision models. It also supports job queues and LLM apps.

Does BentoML collect usage data?

BentoML collects anonymous usage data about its internal API calls only. It excludes user code, model data, model names and stack traces. You can opt out with the --do-not-track CLI option or by setting BENTOML_DO_NOT_TRACK=True.

How does BentoML deploy models?

BentoML packages code, models and dependency configs into a Bento with bentoml build. bentoml containerize then generates a Docker image. You can run that image with Docker or deploy it to Kubernetes.

Does BentoML support GPU inference and batching?

BentoML supports GPU inference and adaptive batching. Its API decorator accepts batchable=True. Services can request GPU resources, such as a specific GPU type.

Sources

BentoML README badge

Maintainers can show the tool's weekly Board rank in their README:

[![Devtoolwatch](https://index.devtoolwatch.com/badge/bentoml-bentoml.svg)](https://index.devtoolwatch.com/tools/bentoml-bentoml/)

How this page is made: Devtoolwatch writes each review from the project's README, releases, package metadata and website, with an AI model, then checks every claim against the source it cites and drops what the sources don't support. The numbers come from Devtoolwatch's daily GitHub snapshots, GitHub's stargazer history, and the npm and PyPI download APIs. Spotted an error? Email [email protected].