Files
droidrun/docs/guides/device-setup.mdx
johnmalek312 5ad3dc1271 fix(ios): remove dead back button branch and update outdated iOS docs
Remove unreachable back button handler in IOSDriver.press_button()
(guard rejects it before reaching the handler). Guard empty
supported_buttons producing malformed system_button description.

Update iOS docs to match current driver behavior: get_date() works,
input_text() clear is supported, swipe uses coordinates, port is 6643.
Replace stub iOS setup guide with full Xcode build/signing/iproxy
instructions.
2026-03-31 23:33:18 +11:00

579 lines
16 KiB
Plaintext

---
title: 'Device Setup'
description: 'Setting up Android and iOS devices for Droidrun automation'
---
## Overview
Droidrun controls devices through a specialized Portal app that bridges your computer and the device.
<Tabs>
<Tab title="Android Setup">
## Prerequisites
<Steps>
<Step title="Install ADB">
**macOS**: `brew install android-platform-tools`
**Linux**: `sudo apt install adb`
**Windows**: Download from [Android Developer Site](https://developer.android.com/studio/releases/platform-tools)
Verify: `adb version`
</Step>
<Step title="Enable USB Debugging">
1. Go to **Settings** > **About phone**
2. Tap **Build number** 7 times (enables Developer options)
3. Go to **Settings** > **Developer options**
4. Enable **USB debugging**
5. Connect device and tap **Always allow**
Verify: `adb devices`
</Step>
<Step title="Install Portal App">
```bash
# Automatic setup (downloads compatible Portal APK)
droidrun setup
# Or specify device
droidrun setup --device SERIAL_NUMBER
```
This will:
- Download the compatible Portal APK for your SDK version
- Install with all permissions granted
- Enable accessibility service automatically
</Step>
<Step title="Verify Setup">
```bash
droidrun ping
# Output: Portal is installed and accessible. You're good to go!
```
</Step>
</Steps>
---
## Portal App
The Droidrun Portal (`com.droidrun.portal`) provides:
- **Accessibility Tree** - Extracts UI elements and their properties
- **Device State** - Tracks current activity, keyboard visibility
- **Action Execution** - Tap, swipe, text input, and other actions
- **Dual Communication** - TCP (faster) or Content Provider (fallback)
<Note>
The Portal only communicates locally via ADB. No data is sent to external servers.
</Note>
---
## Communication Modes
<AccordionGroup>
<Accordion title="TCP Mode (Recommended) - per operation">
**How it works:**
- Portal runs HTTP server on device port 8080
- ADB forwards local port → device port 8080
- Droidrun sends HTTP requests to `localhost:PORT`
**Enable:**
```bash
# CLI
droidrun run "your command" --tcp
# Python
config = DroidConfig(device=DeviceConfig(serial="DEVICE_SERIAL", use_tcp=True))
```
**Troubleshooting:**
```bash
# Check port forwarding
adb forward --list
# Test Portal server
adb shell netstat -an | grep 8080
# Remove all forwards and retry
adb forward --remove-all
droidrun ping --tcp
```
</Accordion>
<Accordion title="Content Provider (Fallback) - per operation">
**How it works:**
- Portal exposes content provider at `content://com.droidrun.portal/`
- Commands sent via ADB shell: `content query --uri ...`
- JSON responses parsed from shell output
**Usage:**
```bash
# Default mode (no flag needed)
droidrun ping
# Python
config = DroidConfig(device=DeviceConfig(serial="DEVICE_SERIAL", use_tcp=False))
```
**Troubleshooting:**
```bash
# Test content provider directly
adb shell content query --uri content://com.droidrun.portal/state
# Should show: Row: 0 result={"data": "{...}"}
```
</Accordion>
</AccordionGroup>
---
## Advanced Setup
<AccordionGroup>
<Accordion title="Wireless Debugging (Android 11+)">
### Setup
<Steps>
<Step title="Enable Wireless Debugging">
1. **Settings** > **Developer options** > **Wireless debugging**
2. Note IP address and port (e.g., `192.168.1.100:37757`)
</Step>
<Step title="Pair Device (First Time)">
**QR Code Method:**
```bash
adb pair <QR_CODE_STRING>
```
**Pairing Code Method:**
1. Tap **Pair device with pairing code**
2. Note pairing code and IP:port
3. Run: `adb pair IP:PORT`
4. Enter pairing code
</Step>
<Step title="Connect">
```bash
adb connect IP:PORT
droidrun ping --device IP:PORT
```
</Step>
</Steps>
### Common Issues
- Connection refused → Check same WiFi network and firewall
- Frequent drops → Use 5GHz WiFi or stay near router
- Can't find IP → Run `adb shell ip addr show wlan0 | grep "inet "` via USB
</Accordion>
<Accordion title="Wireless Debugging (Android 10 and Below)">
<Steps>
<Step title="Enable TCP/IP Mode (USB Required)">
```bash
# Connect via USB first
adb tcpip 5555
```
</Step>
<Step title="Find Device IP">
```bash
adb shell ip addr show wlan0 | grep inet
```
</Step>
<Step title="Connect Wirelessly">
```bash
# Disconnect USB cable
adb connect DEVICE_IP:5555
droidrun ping --device DEVICE_IP:5555
```
</Step>
</Steps>
</Accordion>
<Accordion title="Multiple Devices">
### List Devices
```bash
droidrun devices
# Found 2 connected device(s):
# • emulator-5554
# • 192.168.1.100:5555
```
### Target Specific Device
```bash
# CLI
droidrun run "your command" --device emulator-5554
# Python
config = DroidConfig(device=DeviceConfig(serial="emulator-5554"))
agent = DroidAgent(goal="your task", config=config)
```
### Parallel Control
```python
import asyncio
from droidrun import DeviceConfig, DroidConfig, DroidAgent
from async_adbutils import adb
async def control_device(serial: str, command: str):
device_config = DeviceConfig(serial=serial)
config = DroidConfig(device=device_config)
agent = DroidAgent(goal=command, config=config)
return await agent.run()
async def main():
devices = await adb.list()
tasks = [
control_device(devices[0].serial, "Open settings"),
control_device(devices[1].serial, "Check battery"),
]
results = await asyncio.gather(*tasks)
print(results)
asyncio.run(main())
```
</Accordion>
</AccordionGroup>
---
## Troubleshooting
<AccordionGroup>
<Accordion title="Device not found">
**Symptoms:** `adb devices` shows no devices or `unauthorized`
**Solutions:**
1. Unplug/replug USB cable, try different port
2. Revoke USB debugging authorizations (Developer options)
3. Reconnect and tap "Always allow"
4. Restart ADB: `adb kill-server && adb start-server`
5. **Windows**: Install [Google USB Driver](https://developer.android.com/studio/run/win-usb)
</Accordion>
<Accordion title="Portal not installed">
**Symptoms:** `droidrun ping` fails with "Portal is not installed"
**Solutions:**
1. Reinstall: `droidrun setup`
2. Check: `adb shell pm list packages | grep droidrun`
3. Verify APK architecture matches device (arm64-v8a for most devices)
</Accordion>
<Accordion title="Accessibility service not enabled">
**Symptoms:** `droidrun ping` fails with "accessibility service not enabled"
**Solutions:**
1. Auto-enable:
```bash
adb shell settings put secure enabled_accessibility_services \
com.droidrun.portal/com.droidrun.portal.service.DroidrunAccessibilityService
adb shell settings put secure accessibility_enabled 1
```
2. Manual: Settings > Accessibility > Droidrun Portal > Toggle ON
3. Verify:
```bash
adb shell settings get secure enabled_accessibility_services
# Should contain: com.droidrun.portal/...
```
</Accordion>
<Accordion title="Text input not working">
**Symptoms:** `input_text()` fails or types gibberish
**Solutions:**
1. Keyboard auto-enabled by `AndroidDriver` initialization:
```bash
# Verify
adb shell settings get secure default_input_method
# Should show: com.droidrun.portal/.input.DroidrunKeyboardIME
```
2. Manual switch: Long press space bar → Select "Droidrun Keyboard"
3. Focus the element first (tap it), then input text
</Accordion>
<Accordion title="Empty UI state">
**Symptoms:** `get_state()` returns empty or incomplete UI tree
**Solutions:**
1. Verify accessibility: `droidrun ping`
2. Some apps block accessibility services (WebViews, games, custom UI)
3. Wait for UI: `time.sleep(1)` after tap/swipe
4. Enable Portal overlay to see detected elements
</Accordion>
</AccordionGroup>
</Tab>
<Tab title="iOS Setup (experimental)">
<Warning>
iOS support is currently **experimental**. Functionality is limited compared to Android.
</Warning>
---
## Prerequisites
- **macOS** with **Xcode 15+** installed (iOS Portal must be built from source)
- **Apple Developer account** (free or paid — needed for code signing)
- **iOS device** (iPhone or iPad) with **Developer Mode** enabled
- **USB cable** to connect the device to your Mac (for initial build and deployment)
- **`iproxy`** from `libimobiledevice` (for USB port forwarding)
### Install `iproxy`
```bash
brew install libimobiledevice
```
### Enable Developer Mode on your device
1. Go to **Settings** > **Privacy & Security** > **Developer Mode**
2. Toggle **Developer Mode** on
3. Restart the device when prompted
4. After restart, confirm the prompt to enable Developer Mode
---
## Build and Install the iOS Portal
The iOS Portal is an Xcode project that runs as a UI test — it launches an HTTP server on port **6643** that Droidrun communicates with.
<Steps>
<Step title="Clone the iOS Portal repository">
```bash
git clone https://github.com/droidrun/ios-portal.git
cd ios-portal
```
</Step>
<Step title="Open the project in Xcode">
```bash
open droidrun-ios-portal.xcodeproj
```
</Step>
<Step title="Configure code signing">
You must change the signing team and bundle identifier to your own:
1. In Xcode, select the **droidrun-ios-portal** project in the navigator
2. For **each target** (`droidrun-ios-portal`, `droidrun-ios-portalUITests`):
- Select the target
- Go to the **Signing & Capabilities** tab
- Set **Team** to your Apple Developer account
- Change the **Bundle Identifier** to something unique, e.g.:
- `com.yourname.droidrun-ios-portal`
- `com.yourname.droidrun-ios-portalUITests`
3. Let Xcode automatically manage provisioning profiles
<Warning>
You **must** change the bundle identifiers — the default ones are tied to the Droidrun team's signing certificate and will fail to build on your machine.
</Warning>
</Step>
<Step title="Select your device and run the test">
1. Connect your iOS device via USB
2. In Xcode, select your device from the device dropdown (top bar)
3. If this is your first time deploying to the device, trust the developer certificate:
- On the device: **Settings** > **General** > **VPN & Device Management** > tap your developer profile > **Trust**
4. Run the portal using either method:
**Option A — Xcode UI:**
- Go to **Product** > **Test** (or press `Cmd + U`)
- This builds the app, installs it, and starts the "Droidrun Server" UI test
**Option B — Command line:**
```bash
# Find your device ID
xcrun xctrace list devices
# Run the portal server
./device.sh YOUR_DEVICE_UDID
```
The HTTP server starts on port **6643** and keeps running until the test session ends.
</Step>
<Step title="Set up port forwarding with iproxy">
The portal server runs on the device's localhost. To reach it from your Mac, forward the port over USB:
```bash
iproxy 6643 6643
```
Keep this running in a separate terminal. The portal is now accessible at `http://127.0.0.1:6643`.
<Note>
`iproxy` forwards over USB, so you don't need WiFi connectivity between your Mac and the device.
</Note>
</Step>
<Step title="Verify the connection">
```bash
# Quick health check
curl http://127.0.0.1:6643/device/date
# Should return something like: {"date": "2026-03-31 10:30:00"}
```
Then test with Droidrun:
```bash
droidrun run "take a screenshot" --ios
```
Droidrun will auto-discover the portal on port 6643. If it's on a different port, pass the URL explicitly:
```bash
droidrun run "take a screenshot" --ios --device http://127.0.0.1:6643
```
</Step>
</Steps>
---
## Running on the Simulator
You can also run the portal on the iOS Simulator (no code signing or physical device needed):
```bash
# List available simulators
xcrun simctl list devices available
# Run the portal on a simulator by name
./simulator.sh "iPhone 16 Pro"
```
The simulator portal is accessible at `http://127.0.0.1:6643` directly (no `iproxy` needed).
---
## Architecture
iOS Portal uses a different architecture than Android:
| Feature | Android | iOS |
|---------|---------|-----|
| Communication | ADB + TCP/Content Provider | HTTP server (port 6643) |
| Portal type | APK (auto-installed) | Xcode UI test (built from source) |
| Setup tool | `droidrun setup` | Xcode build + `iproxy` |
| Accessibility | Android Accessibility API | iOS XCUITest framework |
| Text Input | Custom keyboard IME | Direct XCUITest text input |
| Connection | ADB over USB/TCP | USB via `iproxy` or WiFi |
The iOS Portal leverages XCUITest to extract accessibility trees, perform gestures, and capture screenshots — all exposed through a REST API.
---
## Usage
### CLI
```bash
# Auto-discovers portal on port 6643
droidrun run "your command" --ios
# Explicit portal URL
droidrun run "your command" --ios --device http://127.0.0.1:6643
```
### Python API
```python
from droidrun import DroidAgent, DroidConfig, DeviceConfig
config = DroidConfig(
device=DeviceConfig(
platform="ios",
serial="http://127.0.0.1:6643", # optional, auto-discovered if omitted
)
)
agent = DroidAgent(
goal="Open Settings and check WiFi",
config=config
)
result = await agent.run()
```
---
## Supported Features
| Feature | Status | Notes |
|---------|--------|-------|
| `tap()` | ✅ | |
| `swipe()` | ✅ | |
| `input_text()` | ✅ | |
| `screenshot()` | ✅ | |
| `get_state()` | ✅ | Accessibility tree extraction |
| `start_app()` | ✅ | By bundle identifier |
| `get_date()` | ✅ | |
| `press_button()` | ✅ | `home` only |
| `drag()` | ❌ | |
| `install_app()` | ❌ | |
---
## Troubleshooting
<AccordionGroup>
<Accordion title="Build fails with signing error">
**Symptoms:** Xcode shows "Signing for target requires a development team"
**Solutions:**
1. Make sure you changed the **Team** and **Bundle Identifier** for both targets
2. If using a free Apple Developer account, you may need to re-trust the certificate on the device every 7 days (**Settings** > **General** > **VPN & Device Management**)
</Accordion>
<Accordion title="Portal not reachable">
**Symptoms:** `curl http://127.0.0.1:6643/device/date` times out or connection refused
**Solutions:**
1. Verify `iproxy 6643 6643` is running
2. Check the Xcode test is still running (the portal stops when the test ends)
3. Re-run the test with `Cmd + U` in Xcode or `./device.sh UDID`
</Accordion>
<Accordion title="Auto-discovery fails">
**Symptoms:** `droidrun run ... --ios` says "Could not find iOS portal"
**Solutions:**
1. Auto-discovery scans ports 6643-6652. Make sure `iproxy` is forwarding to one of these.
2. Pass the URL explicitly: `--device http://127.0.0.1:6643`
</Accordion>
<Accordion title="Test session keeps stopping">
**Symptoms:** The portal server stops after a few minutes
**Solutions:**
1. Keep Xcode open and the test running — closing Xcode ends the test session
2. Disable auto-lock on the device (**Settings** > **Display & Brightness** > **Auto-Lock** > **Never**)
3. Keep the device plugged in to prevent sleep
</Accordion>
</AccordionGroup>
</Tab>
</Tabs>
---
## Next Steps
- Learn about the [Agent System](/concepts/architecture)
- Explore [Configuration Options](/sdk/configuration)
- Try [Custom Tools](/features/custom-tools)
- Implement [Structured Output](/features/structured-output)