Live preview and take-over

Watch a browser in the console, take control when a task needs a person, and hand it back.

In the console

Every running browser has a live window: a tab strip, an address field, the page, and the element list. Click, type, scroll and press keys directly on the frame; open, switch and close tabs; navigate from the address field. When an Agent run owns the browser the window says so and offers the run; when it does not, the controls are yours.

A standalone, larger view is at /app/browsers/{id}/preview; leaving it does not stop the instance.

What the preview is

A viewport JPEG refreshed by polling — about twice a second while you watch, every two seconds while an agent holds the browser, backing off to every four seconds when the connection is lost, with an elapsed badge and a reconnect control. A scroll, a key or a click at a point is one round trip: the action returns the picture taken right after it. A click on an element first reads the page's observation (refs and rectangles, about 300 ms), which the window fetches ahead while the pointer rests. It is not a video stream and not a recording; see recordings for a durable capture. Reading the preview needs the read scope and does not prolong the browser's idle lifetime.

Input from the window is the same synthetic input the API offers — refs from the observation, click_at for the rest, what you type and paste as insert_text at the page's focus, in the order typed — so what you can do by hand is exactly what the API can do. A click on a recognised text field opens a small input on it, which is also where text that needs an input method (Chinese, Japanese, Korean) is typed; everything else is typed straight onto the page.

Taking over from the agent

An Agent run holds exclusive control of its browser: manual actions answer conflict while it runs. To take over, stop the run (its browser stays alive within its idle and lifetime limits) or wait for it to finish; then act, and start a follow-up run in the same session with browser_id to hand the browser back. Sign-ins done this way are saved into the browser's Profile on stop.

Through the API

GET /v1/browsers/{id}/preview returns the observation plus image (a JPEG data URL), captured_at, viewport and up to 100 target rectangles; each frame_id lasts 30 seconds. ?light=1 returns the picture alone — image, url, title, tabs, a frame_id — in about a tenth of the time; that frame accepts scroll, press, click_at, tab and history actions, and an action by ref needs an observation. POST …/action?frame=1 returns the light frame taken right after the action. Poll it, or use observe when you do not need pixels. Commands to one browser run one at a time and wait their turn; only a full queue answers conflict.