mirror of
https://github.com/droidrun/droidrun.git
synced 2026-05-23 07:40:37 +00:00
Remove unreachable back button handler in IOSDriver.press_button() (guard rejects it before reaching the handler). Guard empty supported_buttons producing malformed system_button description. Update iOS docs to match current driver behavior: get_date() works, input_text() clear is supported, swipe uses coordinates, port is 6643. Replace stub iOS setup guide with full Xcode build/signing/iproxy instructions.
579 lines
16 KiB
Plaintext
579 lines
16 KiB
Plaintext
---
|
|
title: 'Device Setup'
|
|
description: 'Setting up Android and iOS devices for Droidrun automation'
|
|
---
|
|
|
|
## Overview
|
|
|
|
Droidrun controls devices through a specialized Portal app that bridges your computer and the device.
|
|
|
|
<Tabs>
|
|
<Tab title="Android Setup">
|
|
|
|
## Prerequisites
|
|
|
|
<Steps>
|
|
<Step title="Install ADB">
|
|
**macOS**: `brew install android-platform-tools`
|
|
|
|
**Linux**: `sudo apt install adb`
|
|
|
|
**Windows**: Download from [Android Developer Site](https://developer.android.com/studio/releases/platform-tools)
|
|
|
|
Verify: `adb version`
|
|
</Step>
|
|
|
|
<Step title="Enable USB Debugging">
|
|
1. Go to **Settings** > **About phone**
|
|
2. Tap **Build number** 7 times (enables Developer options)
|
|
3. Go to **Settings** > **Developer options**
|
|
4. Enable **USB debugging**
|
|
5. Connect device and tap **Always allow**
|
|
|
|
Verify: `adb devices`
|
|
</Step>
|
|
|
|
<Step title="Install Portal App">
|
|
```bash
|
|
# Automatic setup (downloads compatible Portal APK)
|
|
droidrun setup
|
|
|
|
# Or specify device
|
|
droidrun setup --device SERIAL_NUMBER
|
|
```
|
|
|
|
This will:
|
|
- Download the compatible Portal APK for your SDK version
|
|
- Install with all permissions granted
|
|
- Enable accessibility service automatically
|
|
</Step>
|
|
|
|
<Step title="Verify Setup">
|
|
```bash
|
|
droidrun ping
|
|
# Output: Portal is installed and accessible. You're good to go!
|
|
```
|
|
</Step>
|
|
</Steps>
|
|
|
|
---
|
|
|
|
## Portal App
|
|
|
|
The Droidrun Portal (`com.droidrun.portal`) provides:
|
|
|
|
- **Accessibility Tree** - Extracts UI elements and their properties
|
|
- **Device State** - Tracks current activity, keyboard visibility
|
|
- **Action Execution** - Tap, swipe, text input, and other actions
|
|
- **Dual Communication** - TCP (faster) or Content Provider (fallback)
|
|
|
|
<Note>
|
|
The Portal only communicates locally via ADB. No data is sent to external servers.
|
|
</Note>
|
|
|
|
---
|
|
|
|
## Communication Modes
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="TCP Mode (Recommended) - per operation">
|
|
**How it works:**
|
|
- Portal runs HTTP server on device port 8080
|
|
- ADB forwards local port → device port 8080
|
|
- Droidrun sends HTTP requests to `localhost:PORT`
|
|
|
|
**Enable:**
|
|
```bash
|
|
# CLI
|
|
droidrun run "your command" --tcp
|
|
|
|
# Python
|
|
config = DroidConfig(device=DeviceConfig(serial="DEVICE_SERIAL", use_tcp=True))
|
|
```
|
|
|
|
**Troubleshooting:**
|
|
```bash
|
|
# Check port forwarding
|
|
adb forward --list
|
|
|
|
# Test Portal server
|
|
adb shell netstat -an | grep 8080
|
|
|
|
# Remove all forwards and retry
|
|
adb forward --remove-all
|
|
droidrun ping --tcp
|
|
```
|
|
</Accordion>
|
|
|
|
<Accordion title="Content Provider (Fallback) - per operation">
|
|
**How it works:**
|
|
- Portal exposes content provider at `content://com.droidrun.portal/`
|
|
- Commands sent via ADB shell: `content query --uri ...`
|
|
- JSON responses parsed from shell output
|
|
|
|
**Usage:**
|
|
```bash
|
|
# Default mode (no flag needed)
|
|
droidrun ping
|
|
|
|
# Python
|
|
config = DroidConfig(device=DeviceConfig(serial="DEVICE_SERIAL", use_tcp=False))
|
|
```
|
|
|
|
**Troubleshooting:**
|
|
```bash
|
|
# Test content provider directly
|
|
adb shell content query --uri content://com.droidrun.portal/state
|
|
|
|
# Should show: Row: 0 result={"data": "{...}"}
|
|
```
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
---
|
|
|
|
## Advanced Setup
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Wireless Debugging (Android 11+)">
|
|
### Setup
|
|
|
|
<Steps>
|
|
<Step title="Enable Wireless Debugging">
|
|
1. **Settings** > **Developer options** > **Wireless debugging**
|
|
2. Note IP address and port (e.g., `192.168.1.100:37757`)
|
|
</Step>
|
|
|
|
<Step title="Pair Device (First Time)">
|
|
**QR Code Method:**
|
|
```bash
|
|
adb pair <QR_CODE_STRING>
|
|
```
|
|
|
|
**Pairing Code Method:**
|
|
1. Tap **Pair device with pairing code**
|
|
2. Note pairing code and IP:port
|
|
3. Run: `adb pair IP:PORT`
|
|
4. Enter pairing code
|
|
</Step>
|
|
|
|
<Step title="Connect">
|
|
```bash
|
|
adb connect IP:PORT
|
|
droidrun ping --device IP:PORT
|
|
```
|
|
</Step>
|
|
</Steps>
|
|
|
|
### Common Issues
|
|
|
|
- Connection refused → Check same WiFi network and firewall
|
|
- Frequent drops → Use 5GHz WiFi or stay near router
|
|
- Can't find IP → Run `adb shell ip addr show wlan0 | grep "inet "` via USB
|
|
</Accordion>
|
|
|
|
<Accordion title="Wireless Debugging (Android 10 and Below)">
|
|
<Steps>
|
|
<Step title="Enable TCP/IP Mode (USB Required)">
|
|
```bash
|
|
# Connect via USB first
|
|
adb tcpip 5555
|
|
```
|
|
</Step>
|
|
|
|
<Step title="Find Device IP">
|
|
```bash
|
|
adb shell ip addr show wlan0 | grep inet
|
|
```
|
|
</Step>
|
|
|
|
<Step title="Connect Wirelessly">
|
|
```bash
|
|
# Disconnect USB cable
|
|
adb connect DEVICE_IP:5555
|
|
droidrun ping --device DEVICE_IP:5555
|
|
```
|
|
</Step>
|
|
</Steps>
|
|
</Accordion>
|
|
|
|
<Accordion title="Multiple Devices">
|
|
### List Devices
|
|
|
|
```bash
|
|
droidrun devices
|
|
# Found 2 connected device(s):
|
|
# • emulator-5554
|
|
# • 192.168.1.100:5555
|
|
```
|
|
|
|
### Target Specific Device
|
|
|
|
```bash
|
|
# CLI
|
|
droidrun run "your command" --device emulator-5554
|
|
|
|
# Python
|
|
config = DroidConfig(device=DeviceConfig(serial="emulator-5554"))
|
|
agent = DroidAgent(goal="your task", config=config)
|
|
```
|
|
|
|
### Parallel Control
|
|
|
|
```python
|
|
import asyncio
|
|
from droidrun import DeviceConfig, DroidConfig, DroidAgent
|
|
from async_adbutils import adb
|
|
|
|
async def control_device(serial: str, command: str):
|
|
device_config = DeviceConfig(serial=serial)
|
|
config = DroidConfig(device=device_config)
|
|
agent = DroidAgent(goal=command, config=config)
|
|
return await agent.run()
|
|
|
|
async def main():
|
|
devices = await adb.list()
|
|
|
|
tasks = [
|
|
control_device(devices[0].serial, "Open settings"),
|
|
control_device(devices[1].serial, "Check battery"),
|
|
]
|
|
|
|
results = await asyncio.gather(*tasks)
|
|
print(results)
|
|
|
|
asyncio.run(main())
|
|
```
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Device not found">
|
|
**Symptoms:** `adb devices` shows no devices or `unauthorized`
|
|
|
|
**Solutions:**
|
|
1. Unplug/replug USB cable, try different port
|
|
2. Revoke USB debugging authorizations (Developer options)
|
|
3. Reconnect and tap "Always allow"
|
|
4. Restart ADB: `adb kill-server && adb start-server`
|
|
5. **Windows**: Install [Google USB Driver](https://developer.android.com/studio/run/win-usb)
|
|
</Accordion>
|
|
|
|
<Accordion title="Portal not installed">
|
|
**Symptoms:** `droidrun ping` fails with "Portal is not installed"
|
|
|
|
**Solutions:**
|
|
1. Reinstall: `droidrun setup`
|
|
2. Check: `adb shell pm list packages | grep droidrun`
|
|
3. Verify APK architecture matches device (arm64-v8a for most devices)
|
|
</Accordion>
|
|
|
|
<Accordion title="Accessibility service not enabled">
|
|
**Symptoms:** `droidrun ping` fails with "accessibility service not enabled"
|
|
|
|
**Solutions:**
|
|
1. Auto-enable:
|
|
```bash
|
|
adb shell settings put secure enabled_accessibility_services \
|
|
com.droidrun.portal/com.droidrun.portal.service.DroidrunAccessibilityService
|
|
adb shell settings put secure accessibility_enabled 1
|
|
```
|
|
2. Manual: Settings > Accessibility > Droidrun Portal > Toggle ON
|
|
3. Verify:
|
|
```bash
|
|
adb shell settings get secure enabled_accessibility_services
|
|
# Should contain: com.droidrun.portal/...
|
|
```
|
|
</Accordion>
|
|
|
|
<Accordion title="Text input not working">
|
|
**Symptoms:** `input_text()` fails or types gibberish
|
|
|
|
**Solutions:**
|
|
1. Keyboard auto-enabled by `AndroidDriver` initialization:
|
|
```bash
|
|
# Verify
|
|
adb shell settings get secure default_input_method
|
|
# Should show: com.droidrun.portal/.input.DroidrunKeyboardIME
|
|
```
|
|
2. Manual switch: Long press space bar → Select "Droidrun Keyboard"
|
|
3. Focus the element first (tap it), then input text
|
|
</Accordion>
|
|
|
|
<Accordion title="Empty UI state">
|
|
**Symptoms:** `get_state()` returns empty or incomplete UI tree
|
|
|
|
**Solutions:**
|
|
1. Verify accessibility: `droidrun ping`
|
|
2. Some apps block accessibility services (WebViews, games, custom UI)
|
|
3. Wait for UI: `time.sleep(1)` after tap/swipe
|
|
4. Enable Portal overlay to see detected elements
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
</Tab>
|
|
|
|
<Tab title="iOS Setup (experimental)">
|
|
|
|
<Warning>
|
|
iOS support is currently **experimental**. Functionality is limited compared to Android.
|
|
</Warning>
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
- **macOS** with **Xcode 15+** installed (iOS Portal must be built from source)
|
|
- **Apple Developer account** (free or paid — needed for code signing)
|
|
- **iOS device** (iPhone or iPad) with **Developer Mode** enabled
|
|
- **USB cable** to connect the device to your Mac (for initial build and deployment)
|
|
- **`iproxy`** from `libimobiledevice` (for USB port forwarding)
|
|
|
|
### Install `iproxy`
|
|
|
|
```bash
|
|
brew install libimobiledevice
|
|
```
|
|
|
|
### Enable Developer Mode on your device
|
|
|
|
1. Go to **Settings** > **Privacy & Security** > **Developer Mode**
|
|
2. Toggle **Developer Mode** on
|
|
3. Restart the device when prompted
|
|
4. After restart, confirm the prompt to enable Developer Mode
|
|
|
|
---
|
|
|
|
## Build and Install the iOS Portal
|
|
|
|
The iOS Portal is an Xcode project that runs as a UI test — it launches an HTTP server on port **6643** that Droidrun communicates with.
|
|
|
|
<Steps>
|
|
<Step title="Clone the iOS Portal repository">
|
|
```bash
|
|
git clone https://github.com/droidrun/ios-portal.git
|
|
cd ios-portal
|
|
```
|
|
</Step>
|
|
|
|
<Step title="Open the project in Xcode">
|
|
```bash
|
|
open droidrun-ios-portal.xcodeproj
|
|
```
|
|
</Step>
|
|
|
|
<Step title="Configure code signing">
|
|
You must change the signing team and bundle identifier to your own:
|
|
|
|
1. In Xcode, select the **droidrun-ios-portal** project in the navigator
|
|
2. For **each target** (`droidrun-ios-portal`, `droidrun-ios-portalUITests`):
|
|
- Select the target
|
|
- Go to the **Signing & Capabilities** tab
|
|
- Set **Team** to your Apple Developer account
|
|
- Change the **Bundle Identifier** to something unique, e.g.:
|
|
- `com.yourname.droidrun-ios-portal`
|
|
- `com.yourname.droidrun-ios-portalUITests`
|
|
3. Let Xcode automatically manage provisioning profiles
|
|
|
|
<Warning>
|
|
You **must** change the bundle identifiers — the default ones are tied to the Droidrun team's signing certificate and will fail to build on your machine.
|
|
</Warning>
|
|
</Step>
|
|
|
|
<Step title="Select your device and run the test">
|
|
1. Connect your iOS device via USB
|
|
2. In Xcode, select your device from the device dropdown (top bar)
|
|
3. If this is your first time deploying to the device, trust the developer certificate:
|
|
- On the device: **Settings** > **General** > **VPN & Device Management** > tap your developer profile > **Trust**
|
|
4. Run the portal using either method:
|
|
|
|
**Option A — Xcode UI:**
|
|
- Go to **Product** > **Test** (or press `Cmd + U`)
|
|
- This builds the app, installs it, and starts the "Droidrun Server" UI test
|
|
|
|
**Option B — Command line:**
|
|
```bash
|
|
# Find your device ID
|
|
xcrun xctrace list devices
|
|
|
|
# Run the portal server
|
|
./device.sh YOUR_DEVICE_UDID
|
|
```
|
|
|
|
The HTTP server starts on port **6643** and keeps running until the test session ends.
|
|
</Step>
|
|
|
|
<Step title="Set up port forwarding with iproxy">
|
|
The portal server runs on the device's localhost. To reach it from your Mac, forward the port over USB:
|
|
|
|
```bash
|
|
iproxy 6643 6643
|
|
```
|
|
|
|
Keep this running in a separate terminal. The portal is now accessible at `http://127.0.0.1:6643`.
|
|
|
|
<Note>
|
|
`iproxy` forwards over USB, so you don't need WiFi connectivity between your Mac and the device.
|
|
</Note>
|
|
</Step>
|
|
|
|
<Step title="Verify the connection">
|
|
```bash
|
|
# Quick health check
|
|
curl http://127.0.0.1:6643/device/date
|
|
|
|
# Should return something like: {"date": "2026-03-31 10:30:00"}
|
|
```
|
|
|
|
Then test with Droidrun:
|
|
```bash
|
|
droidrun run "take a screenshot" --ios
|
|
```
|
|
|
|
Droidrun will auto-discover the portal on port 6643. If it's on a different port, pass the URL explicitly:
|
|
```bash
|
|
droidrun run "take a screenshot" --ios --device http://127.0.0.1:6643
|
|
```
|
|
</Step>
|
|
</Steps>
|
|
|
|
---
|
|
|
|
## Running on the Simulator
|
|
|
|
You can also run the portal on the iOS Simulator (no code signing or physical device needed):
|
|
|
|
```bash
|
|
# List available simulators
|
|
xcrun simctl list devices available
|
|
|
|
# Run the portal on a simulator by name
|
|
./simulator.sh "iPhone 16 Pro"
|
|
```
|
|
|
|
The simulator portal is accessible at `http://127.0.0.1:6643` directly (no `iproxy` needed).
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
iOS Portal uses a different architecture than Android:
|
|
|
|
| Feature | Android | iOS |
|
|
|---------|---------|-----|
|
|
| Communication | ADB + TCP/Content Provider | HTTP server (port 6643) |
|
|
| Portal type | APK (auto-installed) | Xcode UI test (built from source) |
|
|
| Setup tool | `droidrun setup` | Xcode build + `iproxy` |
|
|
| Accessibility | Android Accessibility API | iOS XCUITest framework |
|
|
| Text Input | Custom keyboard IME | Direct XCUITest text input |
|
|
| Connection | ADB over USB/TCP | USB via `iproxy` or WiFi |
|
|
|
|
The iOS Portal leverages XCUITest to extract accessibility trees, perform gestures, and capture screenshots — all exposed through a REST API.
|
|
|
|
---
|
|
|
|
## Usage
|
|
|
|
### CLI
|
|
|
|
```bash
|
|
# Auto-discovers portal on port 6643
|
|
droidrun run "your command" --ios
|
|
|
|
# Explicit portal URL
|
|
droidrun run "your command" --ios --device http://127.0.0.1:6643
|
|
```
|
|
|
|
### Python API
|
|
|
|
```python
|
|
from droidrun import DroidAgent, DroidConfig, DeviceConfig
|
|
|
|
config = DroidConfig(
|
|
device=DeviceConfig(
|
|
platform="ios",
|
|
serial="http://127.0.0.1:6643", # optional, auto-discovered if omitted
|
|
)
|
|
)
|
|
|
|
agent = DroidAgent(
|
|
goal="Open Settings and check WiFi",
|
|
config=config
|
|
)
|
|
|
|
result = await agent.run()
|
|
```
|
|
|
|
---
|
|
|
|
## Supported Features
|
|
|
|
| Feature | Status | Notes |
|
|
|---------|--------|-------|
|
|
| `tap()` | ✅ | |
|
|
| `swipe()` | ✅ | |
|
|
| `input_text()` | ✅ | |
|
|
| `screenshot()` | ✅ | |
|
|
| `get_state()` | ✅ | Accessibility tree extraction |
|
|
| `start_app()` | ✅ | By bundle identifier |
|
|
| `get_date()` | ✅ | |
|
|
| `press_button()` | ✅ | `home` only |
|
|
| `drag()` | ❌ | |
|
|
| `install_app()` | ❌ | |
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Build fails with signing error">
|
|
**Symptoms:** Xcode shows "Signing for target requires a development team"
|
|
|
|
**Solutions:**
|
|
1. Make sure you changed the **Team** and **Bundle Identifier** for both targets
|
|
2. If using a free Apple Developer account, you may need to re-trust the certificate on the device every 7 days (**Settings** > **General** > **VPN & Device Management**)
|
|
</Accordion>
|
|
|
|
<Accordion title="Portal not reachable">
|
|
**Symptoms:** `curl http://127.0.0.1:6643/device/date` times out or connection refused
|
|
|
|
**Solutions:**
|
|
1. Verify `iproxy 6643 6643` is running
|
|
2. Check the Xcode test is still running (the portal stops when the test ends)
|
|
3. Re-run the test with `Cmd + U` in Xcode or `./device.sh UDID`
|
|
</Accordion>
|
|
|
|
<Accordion title="Auto-discovery fails">
|
|
**Symptoms:** `droidrun run ... --ios` says "Could not find iOS portal"
|
|
|
|
**Solutions:**
|
|
1. Auto-discovery scans ports 6643-6652. Make sure `iproxy` is forwarding to one of these.
|
|
2. Pass the URL explicitly: `--device http://127.0.0.1:6643`
|
|
</Accordion>
|
|
|
|
<Accordion title="Test session keeps stopping">
|
|
**Symptoms:** The portal server stops after a few minutes
|
|
|
|
**Solutions:**
|
|
1. Keep Xcode open and the test running — closing Xcode ends the test session
|
|
2. Disable auto-lock on the device (**Settings** > **Display & Brightness** > **Auto-Lock** > **Never**)
|
|
3. Keep the device plugged in to prevent sleep
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
- Learn about the [Agent System](/concepts/architecture)
|
|
- Explore [Configuration Options](/sdk/configuration)
|
|
- Try [Custom Tools](/features/custom-tools)
|
|
- Implement [Structured Output](/features/structured-output)
|