Text Chat

The simplest flow on the WebSocket: send a text turn, receive streamed text back, repeat.

Steps

1. Connect to wss://.../user/agents/{agent_id}/ws?token=<jwt>.
2. Send a handshake with conversationType: "text".
3. Wait for the server's handshake_ack.
4. Send a user message.
5. Receive assistant chunks until you see a transaction_end.
6. Send the next user message, or close.

Sending a user turn

{
  "type": "user",
  "content_type": "text",
  "content": "What's the weather in Berlin?"
}

You can also send multimodal turns (text + media):

{
  "type": "user",
  "content_type": "multimodal",
  "content": {
    "text": "What's in this picture?",
    "media": [
      {
        "type": "image",
        "media_id": "med_abc123",
        "s3_key": "media/...",
        "content_type": "image/png",
        "filename": "screenshot.png",
        "size_bytes": 145820
      }
    ]
  }
}

You upload media via the REST API first (POST /user/agents/{agent_id}/media/upload-url), then reference the returned media_id in the message.

What you'll get back

A typical turn looks like:

S → C   { "type": "assistant", "content_type": "text", "content": "It's currently " }
S → C   { "type": "assistant", "content_type": "text", "content": "around 14°C " }
S → C   { "type": "assistant", "content_type": "text", "content": "and partly cloudy." }
S → C   { "type": "transaction_end", "content_type": "text", "content": "done" }

Concatenate the content of consecutive assistant messages to build the full reply. The turn ends when you see transaction_end.

If the agent uses a tool, you'll see additional messages between assistant chunks — see Tool Calls.

Stop / interrupt

There is no client-side "stop" message in text mode. To cut a reply short, just send the next user message — the agent ignores the in-flight reply and starts on your new input.

(In voice mode, you'd use barge_in instead — see Voice Chat.)

Streaming token-by-token vs. chunked

The server streams as the model produces output. Most modern models emit token-level deltas, but the gateway buffers them slightly. Expect chunks of 1–10 tokens each. Render incrementally.

Per-turn metadata

Some assistant messages include metadata:

• metadata.role — usually "assistant".
• metadata.sequence_number — monotonically increasing within a turn. Useful for ordering when out-of-order delivery is a concern.
• metadata.timestamp — server time in milliseconds.

You can ignore these for basic clients; rely on the order messages arrive in.

End of turn

transaction_end marks the end of one assistant turn. After it:

• Send the next user message at any time.
• Or wait — the socket stays open until the TTL.

Tips

• Render assistant chunks immediately rather than waiting for transaction_end. The streaming UX is the whole point.
• Hold off rendering tool messages unless your UI surfaces them — most users only care about the final reply.
• Watch for error between turns; it doesn't always close the socket.

Full example

const ws = new WebSocket(`wss://api.example.com/user/agents/${agentId}/ws?token=${jwt}`);

ws.onopen = () => {
  ws.send(JSON.stringify({
    type: "handshake",
    content_type: "json",
    content: { conversationType: "text" },
  }));
};

let buffer = "";
ws.onmessage = (ev) => {
  if (typeof ev.data !== "string") return;     // ignore binary frames
  const msg = JSON.parse(ev.data);

  switch (msg.type) {
    case "handshake_ack":
      ws.send(JSON.stringify({
        type: "user",
        content_type: "text",
        content: "Hello!",
      }));
      break;

    case "assistant":
      buffer += msg.content;
      render(buffer);
      break;

    case "transaction_end":
      console.log("turn done:", buffer);
      buffer = "";
      break;

    case "error":
      console.error("error:", msg.metadata?.code, msg.content);
      break;
  }
};

ws.onclose = (ev) => {
  console.log("closed:", ev.code, ev.reason);
};