Devtoolwatch BoardMidscene review
Midscene is a vision-based GUI agent and natural-language E2E testing framework for web, mobile, and desktop apps.
TL;DR
Midscene.js is an open-source GUI agent that uses screenshots to act on and verify interfaces described in natural language. It is for developers writing E2E tests across web, Android, iOS, HarmonyOS, and desktop. Tests can be written in YAML with Midscene Test, or the Agent APIs can be added to existing Playwright or Puppeteer tests.
- 15.5kGitHub stars
- +74new stars, week of Oct 25
- ~67.9kdownloads a week, −17% vs 4-week avg
- #14in Browser and computer use this week
Midscene at a glance
- Category
- Browser and computer use
- Maintained by
- Web Infra (organization, @web-infra-dev on GitHub)
- License
- MIT
- Language
- TypeScript, MDX, JavaScript
- Repository
- github.com/web-infra-dev/midscene
- Website
- midscenejs.com
- npm package
- @midscene/web 1.16.2
- Latest release
- v1.16.2 on GitHub, Oct 21, 2026
- First published
- Jul 23, 2024
What Midscene does
- Midscene carries out UI steps described in natural language, locating elements by appearance and position from screenshots instead of selectors.
- Midscene checks visual results such as colors, layout, selection highlights, and content drawn on canvas or shown in native apps.
- Midscene provides Agent APIs: aiAct for autonomous flows, aiTap and aiInput for single actions, aiAssert for assertions, and aiQuery for structured data extraction.
- Midscene Test lets you write test cases in YAML and reuse business logic such as API calls and cleanup through TypeScript Nodes.
- Midscene generates interactive HTML reports showing screenshots, element locations, the AI decision process, and action and assertion results.
- Midscene uses the same Agent APIs across web, Android, iOS, HarmonyOS, macOS, Windows, Linux, and custom interfaces.
- Midscene Skills lets AI coding agents operate applications, verify interfaces, and inspect execution results.
How it works
Midscene takes screenshots, sends them to a configured vision model, and performs clicks, typing, or scrolling. You run it through Midscene Test YAML projects, Playwright, Puppeteer, desktop Chrome, or the JavaScript SDK. Platform adapters connect to Android via ADB, iOS via WebDriverAgent, HarmonyOS NEXT via HDC, and desktop systems with remote RDP support. Supported models include Qwen3.x, Doubao-Seed-2.1, GLM-4.6V, gemini-3.5-flash, and UI-TARS.
How to install and start using Midscene
From npm
npm install -g @midscene/webRequirements
- Node.js 18.19.0 or later for the @midscene/web package.
- Access to a supported vision model, through an API key, a ChatGPT subscription, or a self-hosted open-source model.
- ADB for Android, WebDriverAgent for iOS, or HDC for HarmonyOS NEXT when testing those platforms.
First steps
- Try Midscene with the Chrome extension or a mobile or desktop Playground before writing tests.
- Configure a model using the Model Selection Guide and Model Configuration docs.
- Create a Midscene Test project and write a YAML case using aiAct, aiWaitFor, and aiAssert steps.
- Open the generated HTML report to inspect screenshots, actions, and assertion results.
Who Midscene is for
- Teams adding visual, natural-language steps and assertions to existing Playwright or Puppeteer tests.
- QA engineers testing UIs that are hard to target with selectors, such as canvas, icon-only buttons, or cross-origin iframes.
- Developers testing the same flows across web, mobile, and desktop apps with one API.
- Teams that want AI coding agents to operate and verify application UIs.
Limitations to know
- Midscene requires configuring a vision model, so test runs depend on a model provider or self-hosted model.
- Midscene's official SDK is JavaScript and TypeScript; Python and Java SDKs are community projects.
- Midscene's Jev model support is marked experimental, with documented limitations.
Is Midscene growing? Usage data
In the week of Oct 25–31, 2026, Midscene ranked #348 on the Devtoolwatch Board and #14 in Browser and computer use, with a momentum score of 21 out of 100. Its main package had about 67.9k downloads that week, −17% against its own 4-week average.
Show the weekly numbers
| Week of | New stars |
|---|---|
| Aug 9, 2026 | 66 |
| Aug 16, 2026 | 79 |
| Aug 23, 2026 | 84 |
| Aug 30, 2026 | 60 |
| Sep 6, 2026 | 82 |
| Sep 13, 2026 | 82 |
| Sep 20, 2026 | 83 |
| Sep 27, 2026 | 49 |
| Oct 4, 2026 | 86 |
| Oct 11, 2026 | 179 |
| Oct 18, 2026 | 80 |
| Oct 25, 2026 | 74 |
The Board ranks tools by four signals each week: new stars, new stars for the repo's size, added downloads and download growth beyond the market. How the ranking works.
Midscene alternatives
Other browser and computer-use agents on the Board in the week of Oct 25–31, 2026, by momentum:
| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | ego liteego lite is a Chromium-based browser that lets AI agents like Claude Code or Codex run browser automation using your logged-in browser… | 21k | – |
| #2 | MoliMoli is an open-source headless browser for AI agents, built in Rust, that fetches, extracts, and automates web pages through CLI, CDP,… | 21.9k | – |
| #3 | CuaCua is open-source infrastructure that gives AI agents full desktops, with desktop automation drivers, local VMs, sandboxes, and… | 31.4k | – |
| #4 | Browser HarnessBrowser Harness is an open-source Python tool that lets Claude Code, Codex or Cursor control your real Chrome browser, logged in as you. | 18.8k | 3.3M |
| #5 | Jev UltrafastJev Ultrafast is an open-source Python browser agent that picks an operation and an indexed page element per step, using TypeSafe's Jev. | 24k | – |
| #6 | StagehandStagehand is an SDK for building browser agents that automate and extract data from websites using natural language, in TypeScript,… | 25.9k | 3M |
Recent Midscene releases
Midscene FAQ
Is Midscene free and open source?
Yes. Midscene's source code is on GitHub (web-infra-dev/midscene) under the MIT license, so you can use it for free.
How do I install Midscene?
Run npm install -g @midscene/web. It needs Node.js >=18.19.0.
Is Midscene still maintained?
Yes. The web-infra-dev/midscene repository was last updated on Oct 21, 2026, and the latest release, v1.16.2 on GitHub, came out on Oct 21, 2026.
How popular is Midscene?
Midscene has 15.5k GitHub stars as of Nov 1, 2026. It gained 74 new stars in the week of Oct 25–31, 2026, against a 12-week average of 84 a week. Its main package had about 67.9k downloads that week, −17% against its own 4-week average. That made it #14 among browser and computer-use agents on the Devtoolwatch Board.
What are the best alternatives to Midscene?
Other browser and computer-use agents on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: ego lite (#1), Moli (#2), Cua (#3), Browser Harness (#4), Jev Ultrafast (#5). The alternatives table above has their numbers.
What is Midscene.js?
Midscene.js is an open-source, vision-based GUI agent and E2E testing framework. You describe UI steps and expected results in natural language, and it uses screenshots to act and verify what users see on web, mobile, and desktop apps. It is built by Web Infra, a ByteDance team.
Does Midscene work with Playwright?
Midscene works inside existing Playwright tests. You create a PlaywrightAgent from a Playwright page and call methods like aiAct, aiWaitFor, and aiAssert alongside your existing code, fixtures, and assertions. It also integrates with Puppeteer and desktop Chrome.
Does Midscene need CSS selectors?
Midscene does not need selectors. It locates elements by their appearance and position in screenshots, so it can target icon-only buttons, custom controls, canvas elements, and cross-origin iframes without semantic annotations.
Which AI models does Midscene support?
Midscene supports Qwen3.x, Doubao-Seed-2.1, DeepSeek V4 Flash, GLM-4.6V, gemini-3.5-flash, and UI-TARS, including open-source options you can self-host. Version 1.16 added support for Jev 1.4, and the docs also cover using a ChatGPT subscription.
Which platforms can Midscene test?
Midscene can test web apps through Playwright, Puppeteer, or desktop Chrome. It also covers Android via ADB, iOS via WebDriverAgent, HarmonyOS NEXT via HDC, and macOS, Windows, and Linux desktops. Any custom interface works if you provide screenshots and action capabilities.
Can Midscene run in CI?
Midscene Test can run in CI. The documentation includes a guide for running Midscene Test in GitHub Actions. Version 1.16.1 added file concurrency through a maxConcurrentFiles option.
Sources
Midscene README badge
Maintainers can show the tool's weekly Board rank in their README:
[](https://index.devtoolwatch.com/tools/midscene/)How this page is made: Devtoolwatch writes each review from the project's README, releases, package metadata and website, with an AI model, then checks every claim against the source it cites and drops what the sources don't support. The numbers come from Devtoolwatch's daily GitHub snapshots, GitHub's stargazer history, and the npm and PyPI download APIs. Spotted an error? Email [email protected].