# Live stream API

> The device WebSocket - authentication with a key, cookie or single-use stream token, binary JPEG frames, JSON events, input messages and limits.

Source: https://akaphone-docs.pages.dev/api/stream/

Each device has a WebSocket that carries its screen as JPEG frames, events about the device and its tasks, and input from you. The dashboard's live view is built on it. Background in [Live streaming](https://akaphone-docs.pages.dev/concepts/streaming/).

## Connect

`WS /v1/devices/:id/stream`

```text
wss://akaphone.abhgnx-417.workers.dev/v1/devices/dev_s6zr7mr4f2f1/stream
```

A plain HTTP request (no WebSocket upgrade) gets `426` with code `invalid_request`. You can connect to a device in any status; frames flow only while it is `running`.

### Authentication

Use one of:

| Method | Use it from |
|---|---|
| `Authorization: Bearer ak_live_…` header on the upgrade request | Servers, scripts, Node.js and Python clients. |
| `?key=<stream token>` query parameter (`?token=` also works) | Browsers. Get the token from your server with [`POST /v1/devices/:id/stream-token`](https://akaphone-docs.pages.dev/api/devices/#create-a-stream-token). It is valid for 5 minutes, for this device only, and for a single connection. |
| The dashboard's session cookie | The AkaPhone dashboard only. |

Failures are returned before the upgrade, as normal JSON errors: `401 unauthorized` (no or bad credentials, or a used or expired token), `403 forbidden` (suspended account), `404 not_found` (not your device).

## Server to client

### Frames

**Binary messages** are complete JPEG images of the screen, with no prefix or header. Draw each one as it arrives.

- Rate: 2 fps on Free, 4 on Dev, 5 on Team.
- Size: downscaled by the host, 540 px wide by default; scale input coordinates back to device pixels.
- On connect you get the most recent frame straight away if there is one. An unchanged screen is resent every 10 seconds.

### Events

**Text messages** are JSON objects with a `t` field:

| `t` | Fields | When |
|---|---|---|
| `status` | `device` (the [device object](https://akaphone-docs.pages.dev/api/devices/#the-device-object)) | On connect, and whenever the device's status changes. |
| `task.started` | `task_id` | ARTEMIS started a task. |
| `task.step` | `task_id`, `n`, `label`, `ms` | Each step of a running task. |
| `task.done` | `task` (the [task object](https://akaphone-docs.pages.dev/api/tasks/#the-task-object), without `steps`) | A task finished: succeeded, failed, cancelled or timed out. |
| `app.installed` | `device_id`, `install_id`, `package`, `error` | An [APK install](https://akaphone-docs.pages.dev/api/devices/#install-an-app) finished (also available from [get an install](https://akaphone-docs.pages.dev/api/devices/#get-an-install)). `error` is `null` on success; `package` may be `null` if it could not be determined. |
| `credits.low` | `balance` | Your balance is 5 credits or less. |
| `device.stopped` | `reason`: `user`, `idle`, `credits` or `lost` | The device is stopping or has stopped. No more frames will come; close the socket. |
| `error` | `error: { code, message }` | Your last input was rejected: `rate_limited` or `invalid_request`. |

```json
{"t":"status","device":{"id":"dev_s6zr7mr4f2f1","status":"running","…":"…"}}
{"t":"task.started","task_id":"tsk_01m5vqsywgehe4jhva3g0r"}
{"t":"task.step","task_id":"tsk_01m5vqsywgehe4jhva3g0r","n":1,"label":"Open the Settings app","ms":2140}
{"t":"task.done","task":{"id":"tsk_01m5vqsywgehe4jhva3g0r","status":"succeeded","result":"Android version: 14","…":"…"}}
{"t":"app.installed","device_id":"dev_s6zr7mr4f2f1","install_id":"ins_01m5vqsywg99189k5xj7f5","package":"com.example.shop","error":null}
{"t":"credits.low","balance":4}
{"t":"device.stopped","reason":"credits"}
```

Ignore event types you do not recognise; new ones may be added.

## Client to server

### Sending input

Send input as a text message with `t: "input"` and the same fields as [`POST /v1/devices/:id/input`](https://akaphone-docs.pages.dev/api/devices/#send-input):

```json
{"t":"input","type":"tap","x":360,"y":780}
{"t":"input","type":"swipe","x1":360,"y1":1200,"x2":360,"y2":400,"ms":250}
{"t":"input","type":"text","text":"hello@example.com"}
{"t":"input","type":"key","key":"BACK"}
```

- Coordinates are device pixels (720×1560 for `pixel_8`), not frame pixels.
- `text` is at most 500 characters per message. Keys: `HOME`, `BACK`, `RECENTS`, `ENTER`, `DEL`, `TAB`.
- At most **20 messages per second per connection**; extra messages are dropped and you get an `error` event.
- Input is ignored while the device is not `running`. Any other message type is ignored.

## Keeping the device alive

An open connection counts as activity: a device with a viewer is never stopped as idle. It still stops when you stop it or your credits run out. Close sockets you are not using.

## Examples

```typescript title="Browser"
// 1. Your server: POST /v1/devices/:id/stream-token with your API key, return { token } to the page.
const { token } = await fetch(`/my-backend/stream-token?device=${deviceId}`).then((r) => r.json());

// 2. The page: connect with the token, draw frames on a canvas.
const ws = new WebSocket(`wss://akaphone.abhgnx-417.workers.dev/v1/devices/${deviceId}/stream?key=${encodeURIComponent(token)}`);
ws.binaryType = "arraybuffer";
const canvas = document.querySelector("canvas")!;
const ctx = canvas.getContext("2d")!;

ws.onmessage = async (msg) => {
  if (typeof msg.data !== "string") {
    const img = await createImageBitmap(new Blob([msg.data], { type: "image/jpeg" }));
    canvas.width = img.width;
    canvas.height = img.height;
    ctx.drawImage(img, 0, 0);
    img.close();
    return;
  }
  const ev = JSON.parse(msg.data);
  if (ev.t === "task.step") console.log(`step ${ev.n}: ${ev.label}`);
  if (ev.t === "device.stopped") ws.close();
};

// Tap where the user clicked, scaled from frame pixels to device pixels (720 wide for pixel_8).
canvas.addEventListener("click", (e) => {
  const r = canvas.getBoundingClientRect();
  const scale = 720 / r.width;
  ws.send(JSON.stringify({ t: "input", type: "tap", x: Math.round((e.clientX - r.left) * scale), y: Math.round((e.clientY - r.top) * scale) }));
});
```
```typescript title="Node.js"
// npm install ws
import WebSocket from "ws";
import { writeFileSync } from "node:fs";

const ws = new WebSocket("wss://akaphone.abhgnx-417.workers.dev/v1/devices/dev_s6zr7mr4f2f1/stream", {
  headers: { Authorization: `Bearer ${process.env.AKAPHONE_API_KEY}` },
});

let frames = 0;
ws.on("message", (data, isBinary) => {
  if (isBinary) {
    writeFileSync(`frame-${String(frames++).padStart(4, "0")}.jpg`, data as Buffer);
    return;
  }
  const ev = JSON.parse(data.toString());
  console.log(ev.t, ev);
  if (ev.t === "device.stopped") ws.close();
});
ws.on("open", () => ws.send(JSON.stringify({ t: "input", type: "key", key: "HOME" })));
```
```python title="Python"
# pip install "websockets>=14"
import asyncio, json, os
import websockets

async def main():
    url = "wss://akaphone.abhgnx-417.workers.dev/v1/devices/dev_s6zr7mr4f2f1/stream"
    headers = {"Authorization": f"Bearer {os.environ['AKAPHONE_API_KEY']}"}
    async with websockets.connect(url, additional_headers=headers, max_size=None) as ws:
        await ws.send(json.dumps({"t": "input", "type": "key", "key": "HOME"}))
        async for msg in ws:
            if isinstance(msg, bytes):
                open("latest.jpg", "wb").write(msg)
                continue
            ev = json.loads(msg)
            print(ev["t"], ev)
            if ev["t"] == "device.stopped":
                break

asyncio.run(main())
```

## Reconnecting

Connections can drop (network changes, deploys). Reconnect with exponential backoff, and get a **new** stream token for each attempt if you use tokens. On reconnect you receive a fresh `status` event and the latest frame, so there is no state to replay. Stop reconnecting after `device.stopped`.