# ESP32 iPhone Agent review

> ESP32 iPhone Agent lets an AI vision model operate a real iPhone through an ESP32-S3 board acting as AirPlay receiver and USB mouse.

Devtoolwatch Board, the week of Oct 25–31, 2026. Canonical page: https://index.devtoolwatch.com/tools/onehumanbeing-esp32-iphone-agent/

## TL;DR

ESP32 iPhone Agent lets an AI model use a real iPhone through an ESP32-S3 board that receives AirPlay mirroring and acts as a USB mouse and keyboard. The agent runs in an iPhone app on the same phone. It is for developers who want phone automation without a jailbreak or a computer at run time, using any OpenAI-compatible vision model.

## ESP32 iPhone Agent at a glance

- **Category:** [Coding agents](https://index.devtoolwatch.com/categories/coding-agents/)
- **Maintained by:** Henry Yuan (individual, [@onehumanbeing](https://github.com/onehumanbeing) on GitHub)
- **License:** MIT
- **Language:** C, Swift, Python
- **Repository:** [github.com/onehumanbeing/esp32-iphone-agent](https://github.com/onehumanbeing/esp32-iphone-agent)
- **Website:** [handsy.tech?ref=github](https://handsy.tech?ref=github)
- **Latest release:** [v0.1.0](https://github.com/onehumanbeing/esp32-iphone-agent/releases/tag/v0.1.0) on GitHub, Oct 14, 2026
- **First published:** Oct 11, 2026

## What ESP32 iPhone Agent does

- ESP32 iPhone Agent mirrors the iPhone screen over AirPlay to the board, which forwards the H.264 stream to the app over BLE.
- ESP32 iPhone Agent taps, swipes, draws, types and presses keys through the board acting as a USB HID mouse and keyboard.
- ESP32 iPhone Agent sends screenshots to any OpenAI-compatible vision model and runs a planner and an actor, with optional separate models.
- ESP32 iPhone Agent tracks the relative pointer, re-homes it and corrects missed taps before precise actions.
- ESP32 iPhone Agent pauses on a lost board, hotspot or mirroring link and resumes from the same step.
- ESP32 iPhone Agent keeps every run on the phone for step-by-step replay and trace export for review by another agent.
- ESP32 iPhone Agent can run saved tasks from Siri, Shortcuts, widgets and Control Center.

### How it works

The ESP32-S3 board opens its own Wi-Fi hotspot and appears as an AirPlay Screen Mirroring target. It passes the H.264 stream to the iPhone app over BLE. The app decodes it, queries an OpenAI-compatible vision model over cellular data, and sends actions back over BLE. The board performs them as a USB mouse and keyboard, with AssistiveTouch showing the pointer. The phone powers the board.

## How to install ESP32 iPhone Agent

**Install the BLE library**

```
arduino-cli lib install "NimBLE-Arduino@2.5.1"
```

**Flash the board**

```
hardware/flash.sh
```

**Open the iOS project**

```
open ios/IPhoneAgent.xcodeproj
```

### Requirements

- An ESP32-S3 board with native USB on its USB-C port and PSRAM, such as the Seeed Studio XIAO ESP32S3 (8 MB PSRAM).
- An iPhone on iOS 18 or later with cellular data; Lightning models need a Lightning to USB Camera Adapter.
- An OpenAI-compatible chat API that accepts images, cloud or self-hosted, with an endpoint and key.
- A Mac with Xcode to flash the board and build the app once; a free Apple ID can sign it.

### First steps

1. Flash the ESP32-S3 from a Mac with arduino-cli, the ESP32 core 3.3.x and NimBLE-Arduino 2.5.x, always using flash.sh.
2. Copy Local.xcconfig.example, set DEVELOPMENT_TEAM and IA_BUNDLE_ID, then run the IPhoneAgent scheme on your iPhone from Xcode.
3. Plug the board into the iPhone and follow the app's four setup steps: Bluetooth pairing, Screen Mirroring, AssistiveTouch, and model endpoint.
4. Type a goal such as "Open Settings and go to Wi-Fi" and press Start.

## Who ESP32 iPhone Agent is for

- Developers automating tasks on a real iPhone without a jailbreak, Developer Mode or a computer at run time.
- Teams running their own vision models through vLLM, SGLang, llama.cpp or Ollama.
- Researchers who want exportable traces of screenshots and actions for agent review.

### Limitations

- ESP32 iPhone Agent mirrors the screen small (480×1040, at most 5 fps) over BLE; each step takes a few seconds.
- ESP32 iPhone Agent cannot see passcode entry, Apple Pay or DRM video, and mirroring stops when the phone locks.
- ESP32 iPhone Agent supports ASCII keyboard input only and does not yet support landscape apps.
- ESP32 iPhone Agent apps signed with a free Apple ID must be re-signed every 7 days; one-tap hotspot join needs a paid account.

## Usage data (the week of Oct 25–31, 2026)

- Board rank: #25 overall, #14 in Coding agents, momentum score 86
- New GitHub stars per week (last 12 weeks): Aug 9: n/a; Aug 16: n/a; Aug 23: n/a; Aug 30: n/a; Sep 6: n/a; Sep 13: n/a; Sep 20: n/a; Sep 27: n/a; Oct 4: n/a; Oct 11: 3k; Oct 18: 795; Oct 25: 345

## ESP32 iPhone Agent alternatives

| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | [Cindy](https://index.devtoolwatch.com/tools/makecindy-cindy/) | 8.8k | – |
| #2 | [OpenMausBot](https://index.devtoolwatch.com/tools/milind-soni-openmausbot/) | 9.6k | – |
| #3 | [Orca](https://index.devtoolwatch.com/tools/stablyai-orca/) | 111.7k | – |
| #4 | [OpenBot](https://index.devtoolwatch.com/tools/copilotkit-openbot/) | 9.1k | – |
| #5 | [Agent Reach](https://index.devtoolwatch.com/tools/panniantong-agent-reach/) | 107.2k | – |
| #6 | [Skill Recorder](https://index.devtoolwatch.com/tools/microsoft-skill-recorder/) | 5.6k | – |

## ESP32 iPhone Agent FAQ

### Is ESP32 iPhone Agent free and open source?

Yes. ESP32 iPhone Agent's source code is on GitHub (onehumanbeing/esp32-iphone-agent) under the MIT license, so you can use it for free. Its website also links to a pricing page (https://handsy.tech/pricing) for any paid plans.

### How do I install ESP32 iPhone Agent?

Run `arduino-cli lib install "NimBLE-Arduino@2.5.1"`. The install section above has the other options from the project's README.

### Is ESP32 iPhone Agent still maintained?

Yes. The onehumanbeing/esp32-iphone-agent repository was last updated on Oct 28, 2026, and the latest release, v0.1.0 on GitHub, came out on Oct 14, 2026.

### How popular is ESP32 iPhone Agent?

ESP32 iPhone Agent has 4.2k GitHub stars as of Nov 1, 2026. It gained 345 new stars in the week of Oct 25–31, 2026, against a 12-week average of 1.4k a week. That made it #14 among AI coding agents on the Devtoolwatch Board.

### What are the best alternatives to ESP32 iPhone Agent?

Other AI coding agents on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: Cindy (#1), OpenMausBot (#2), Orca (#3), OpenBot (#4), Agent Reach (#5). The alternatives table above has their numbers.

### Does ESP32 iPhone Agent need a computer while it runs?

No. ESP32 iPhone Agent only needs a Mac with Xcode once to flash the board and build the app. At run time the board handles mirroring and input, and the agent runs in an iPhone app on the same phone.

### Which models work with ESP32 iPhone Agent?

ESP32 iPhone Agent works with any OpenAI-compatible vision endpoint, including vLLM, SGLang, llama.cpp, Ollama and cloud APIs. You pick a model from the endpoint's list or enter a model ID, and can set separate planner and actor models.

### How does ESP32 iPhone Agent reach the model if Wi-Fi is used by the board?

ESP32 iPhone Agent reaches the model over cellular data. While the agent runs, the phone's Wi-Fi is joined to the board's hotspot, which has no internet access.

### Can I mirror from a Mac to ESP32 iPhone Agent's board?

No. ESP32 iPhone Agent requires an iPhone as the AirPlay source. With macOS Screen Mirroring, the handshake can look healthy while the board never receives video.

### Can I customize the agent's prompts in ESP32 iPhone Agent?

Yes. ESP32 iPhone Agent keeps its planner and actor prompts as plain text in the prompts/ folder, bundled with the app. Edit them and rebuild the app.

### Is the older computer-based version of ESP32 iPhone Agent still available?

Yes. ESP32 iPhone Agent keeps the original Mac/Linux setup with UxPlay and a Python server as tag v0.1.0 and the legacy branch.

## Sources

1. [onehumanbeing/esp32-iphone-agent on GitHub](https://github.com/onehumanbeing/esp32-iphone-agent)
2. [ESP32 iPhone Agent README](https://github.com/onehumanbeing/esp32-iphone-agent/blob/main/README.md)
3. [ESP32 iPhone Agent website](https://handsy.tech?ref=github)
4. [Release v0.1.0 (Oct 14, 2026)](https://github.com/onehumanbeing/esp32-iphone-agent/releases/tag/v0.1.0)
5. [Devtoolwatch Board, the week of Oct 25–31, 2026](https://index.devtoolwatch.com/)