Giving an LLM Eyes and Hands on a Mobile Simulator

Tapflow turned its existing mobile simulator relay APIs into an MCP server with 13 tools for LLM control.

An MCP-compatible, vision-capable model can now inspect screenshots, tap, swipe, type, install apps, and launch apps in simulators.

The system uses pixel-space screenshots for easier reasoning, normalizes touch input internally, and handles some actions asynchronously over WebSocket.

The approach shows how existing APIs can power LLM-driven mobile UI testing and automation without a new control layer.