Class Actions

Use natural language instructions to perform UI operations on the box. The endpoint will stream progress events before and after the action is executed. If you don't need intermediate events, set stream to false.

Click

Drag

Extract data from the UI interface using a JSON schema.

Perform a long press action at specified coordinates for a specified duration. Useful for triggering context menus, drag operations, or other long-press interactions.

Move to position

Press device buttons like power, volume, home, back, etc.

Simulates pressing a specific key by triggering the complete keyboard key event chain (keydown, keypress, keyup). Use this to activate keyboard key event listeners such as shortcuts or form submissions.

Start recording the box screen. Only one recording can be active at a time. If a recording is already in progress, starting a new recording will stop the previous one and keep only the latest recording.

Stop recording the box screen

Disable the device's background screen rewind recording.

Enable the device's background screen rewind recording.

Rewind and capture the device's background screen recording from a specified time period.

Get the current structured screen layout information. This endpoint returns detailed structural information about the UI elements currently displayed on the screen, which can be used for UI automation, element analysis, and accessibility purposes. The format varies by box type: Android boxes return XML format with detailed UI hierarchy information including element bounds, text content, resource IDs, and properties, while other box types may return different structured formats.

Rotate the screen orientation. Note that even after rotating the screen, applications or system layouts may not automatically adapt to the gravity sensor changes, so visual changes may not always occur.

Take screenshot

Performs a scroll action. Supports both advanced scroll with coordinates and simple scroll with direction.

Get the box action settings

Reset the box settings to default

Update the box action settings

Performs a swipe in the specified direction

Tap action for Android devices using ADB input tap command

Touch

Directly inputs text content without triggering physical key events (keydown, etc.), ideal for quickly filling large amounts of text when intermediate input events aren't needed.