how it works

the path of one tap

prompt ↓ fine-tuned VLM (UI-TARS-2, on-device via MLX) ↓ typed action → persistent XCUITest runner → iPhone ↓ accessibility tree + screenshot ↓ back into the loop

No jailbreak, no Mac required, no iPhone modifications. The agent runs on a host (Mac / Linux / cloud); the iPhone is a stock production device.

what's verifiable from outside

AX fetch

40–300 ms

screenshot

30–150 ms

tap injection

0.1–5 ms

apps in corpus

47+

Numbers are p50 ranges on physical iPhones over the local network. Simulator is faster on every axis.

what's real in the demo

Everything except the live voice commands. Real iPhone 15 Pro, real iOS 26.4, real App Store apps, real ClankDriver taps and swipes, single-take recording. The Dynamic Island is rendered by SpringBoard — iOS, not the app — so it can't be composited in post.

The voice was for presentation. At the start of the video I fed the agent the same list of tasks as a typed prompt; the spoken voiceover is just so a viewer can follow along. Replace my voice with a chat input, an API call, or a webhook payload and the rest runs identically.

what clank can do today

Drive any iOS app on a stock iPhone or iPad — no SDK, no developer integration, no App-Intent registration.
Single natural-language prompt → multi-step execution across apps (e.g., "find a Thai place near me, text Mark the address" → search, switch app, compose, send).
Cross-app workflows: Messages → Photos → Mail → Safari.
Record-and-replay with self-heal when the UI shifts — scripts survive UI churn instead of failing.
Multi-device fleets: same task run in parallel across many phones (10+ devices in CI).
Recovery from unexpected state — modals, permissions, auth prompts, system alerts.
Apps without active anti-debug protections.
Reachable over WiFi or a tunnel — host can be a local Mac, Linux box, or cloud.

what it can't do yet

Unlock the phone.

what's under NDA

The iOS-internals stack, the training pipeline + trace corpus, the synthetic-iOS substrate (ClankGym) and its sim2real fidelity measurements, the tether path and signing infrastructure, the runtime that closes the latency gap to milliseconds, and the Y2 hardware + multi-device-class roadmap.