# Midscene review

> Midscene is a vision-based GUI agent and natural-language E2E testing framework for web, mobile, and desktop apps.

Devtoolwatch Board, the week of Oct 25–31, 2026. Canonical page: https://index.devtoolwatch.com/tools/midscene/

## TL;DR

Midscene.js is an open-source GUI agent that uses screenshots to act on and verify interfaces described in natural language. It is for developers writing E2E tests across web, Android, iOS, HarmonyOS, and desktop. Tests can be written in YAML with Midscene Test, or the Agent APIs can be added to existing Playwright or Puppeteer tests.

## Midscene at a glance

- **Category:** [Browser and computer use](https://index.devtoolwatch.com/categories/browser-and-computer-use/)
- **Maintained by:** Web Infra (organization, [@web-infra-dev](https://github.com/web-infra-dev) on GitHub)
- **License:** MIT
- **Language:** TypeScript, MDX, JavaScript
- **Repository:** [github.com/web-infra-dev/midscene](https://github.com/web-infra-dev/midscene)
- **Website:** [midscenejs.com](https://midscenejs.com)
- **npm package:** [@midscene/web](https://www.npmjs.com/package/@midscene/web) 1.16.2
- **Latest release:** [v1.16.2](https://github.com/web-infra-dev/midscene/releases/tag/v1.16.2) on GitHub, Oct 21, 2026
- **First published:** Jul 23, 2024

## What Midscene does

- Midscene carries out UI steps described in natural language, locating elements by appearance and position from screenshots instead of selectors.
- Midscene checks visual results such as colors, layout, selection highlights, and content drawn on canvas or shown in native apps.
- Midscene provides Agent APIs: aiAct for autonomous flows, aiTap and aiInput for single actions, aiAssert for assertions, and aiQuery for structured data extraction.
- Midscene Test lets you write test cases in YAML and reuse business logic such as API calls and cleanup through TypeScript Nodes.
- Midscene generates interactive HTML reports showing screenshots, element locations, the AI decision process, and action and assertion results.
- Midscene uses the same Agent APIs across web, Android, iOS, HarmonyOS, macOS, Windows, Linux, and custom interfaces.
- Midscene Skills lets AI coding agents operate applications, verify interfaces, and inspect execution results.

### How it works

Midscene takes screenshots, sends them to a configured vision model, and performs clicks, typing, or scrolling. You run it through Midscene Test YAML projects, Playwright, Puppeteer, desktop Chrome, or the JavaScript SDK. Platform adapters connect to Android via ADB, iOS via WebDriverAgent, HarmonyOS NEXT via HDC, and desktop systems with remote RDP support. Supported models include Qwen3.x, Doubao-Seed-2.1, GLM-4.6V, gemini-3.5-flash, and UI-TARS.

## How to install Midscene

**From npm**

```
npm install -g @midscene/web
```

### Requirements

- Node.js 18.19.0 or later for the @midscene/web package.
- Access to a supported vision model, through an API key, a ChatGPT subscription, or a self-hosted open-source model.
- ADB for Android, WebDriverAgent for iOS, or HDC for HarmonyOS NEXT when testing those platforms.

### First steps

1. Try Midscene with the Chrome extension or a mobile or desktop Playground before writing tests.
2. Configure a model using the Model Selection Guide and Model Configuration docs.
3. Create a Midscene Test project and write a YAML case using aiAct, aiWaitFor, and aiAssert steps.
4. Open the generated HTML report to inspect screenshots, actions, and assertion results.

## Who Midscene is for

- Teams adding visual, natural-language steps and assertions to existing Playwright or Puppeteer tests.
- QA engineers testing UIs that are hard to target with selectors, such as canvas, icon-only buttons, or cross-origin iframes.
- Developers testing the same flows across web, mobile, and desktop apps with one API.
- Teams that want AI coding agents to operate and verify application UIs.

### Limitations

- Midscene requires configuring a vision model, so test runs depend on a model provider or self-hosted model.
- Midscene's official SDK is JavaScript and TypeScript; Python and Java SDKs are community projects.
- Midscene's Jev model support is marked experimental, with documented limitations.

## Usage data (the week of Oct 25–31, 2026)

- Board rank: #348 overall, #14 in Browser and computer use, momentum score 21
- Downloads that week: 67.9k (−17% vs 4-week average)
- New GitHub stars per week (last 12 weeks): Aug 9: 66; Aug 16: 79; Aug 23: 84; Aug 30: 60; Sep 6: 82; Sep 13: 82; Sep 20: 83; Sep 27: 49; Oct 4: 86; Oct 11: 179; Oct 18: 80; Oct 25: 74

## Midscene alternatives

| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | [ego lite](https://index.devtoolwatch.com/tools/citrolabs-ego-lite/) | 21k | – |
| #2 | [Moli](https://index.devtoolwatch.com/tools/lexmount-moli/) | 21.9k | – |
| #3 | [Cua](https://index.devtoolwatch.com/tools/cua/) | 31.4k | – |
| #4 | [Browser Harness](https://index.devtoolwatch.com/tools/browser-use-browser-harness/) | 18.8k | 3.3M |
| #5 | [Jev Ultrafast](https://index.devtoolwatch.com/tools/browser-use-jev-ultrafast/) | 24k | – |
| #6 | [Stagehand](https://index.devtoolwatch.com/tools/stagehand/) | 25.9k | 3M |

## Midscene FAQ

### Is Midscene free and open source?

Yes. Midscene's source code is on GitHub (web-infra-dev/midscene) under the MIT license, so you can use it for free.

### How do I install Midscene?

Run `npm install -g @midscene/web`. It needs Node.js >=18.19.0.

### Is Midscene still maintained?

Yes. The web-infra-dev/midscene repository was last updated on Oct 21, 2026, and the latest release, v1.16.2 on GitHub, came out on Oct 21, 2026.

### How popular is Midscene?

Midscene has 15.5k GitHub stars as of Nov 1, 2026. It gained 74 new stars in the week of Oct 25–31, 2026, against a 12-week average of 84 a week. Its main package had about 67.9k downloads that week, −17% against its own 4-week average. That made it #14 among browser and computer-use agents on the Devtoolwatch Board.

### What are the best alternatives to Midscene?

Other browser and computer-use agents on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: ego lite (#1), Moli (#2), Cua (#3), Browser Harness (#4), Jev Ultrafast (#5). The alternatives table above has their numbers.

### What is Midscene.js?

Midscene.js is an open-source, vision-based GUI agent and E2E testing framework. You describe UI steps and expected results in natural language, and it uses screenshots to act and verify what users see on web, mobile, and desktop apps. It is built by Web Infra, a ByteDance team.

### Does Midscene work with Playwright?

Midscene works inside existing Playwright tests. You create a PlaywrightAgent from a Playwright page and call methods like aiAct, aiWaitFor, and aiAssert alongside your existing code, fixtures, and assertions. It also integrates with Puppeteer and desktop Chrome.

### Does Midscene need CSS selectors?

Midscene does not need selectors. It locates elements by their appearance and position in screenshots, so it can target icon-only buttons, custom controls, canvas elements, and cross-origin iframes without semantic annotations.

### Which AI models does Midscene support?

Midscene supports Qwen3.x, Doubao-Seed-2.1, DeepSeek V4 Flash, GLM-4.6V, gemini-3.5-flash, and UI-TARS, including open-source options you can self-host. Version 1.16 added support for Jev 1.4, and the docs also cover using a ChatGPT subscription.

### Which platforms can Midscene test?

Midscene can test web apps through Playwright, Puppeteer, or desktop Chrome. It also covers Android via ADB, iOS via WebDriverAgent, HarmonyOS NEXT via HDC, and macOS, Windows, and Linux desktops. Any custom interface works if you provide screenshots and action capabilities.

### Can Midscene run in CI?

Midscene Test can run in CI. The documentation includes a guide for running Midscene Test in GitHub Actions. Version 1.16.1 added file concurrency through a maxConcurrentFiles option.

## Sources

1. [web-infra-dev/midscene on GitHub](https://github.com/web-infra-dev/midscene)
2. [Midscene README](https://github.com/web-infra-dev/midscene/blob/main/README.md)
3. [@midscene/web on npm](https://www.npmjs.com/package/@midscene/web)
4. [Midscene website](https://midscenejs.com)
5. [Release v1.16.2 (Oct 21, 2026)](https://github.com/web-infra-dev/midscene/releases/tag/v1.16.2)
6. [Release v1.16.1 (Oct 21, 2026)](https://github.com/web-infra-dev/midscene/releases/tag/v1.16.1)
7. [Release v1.16.0 (Oct 16, 2026)](https://github.com/web-infra-dev/midscene/releases/tag/v1.16.0)
8. [Devtoolwatch Board, the week of Oct 25–31, 2026](https://index.devtoolwatch.com/)