WebSocket API Overview

Stream Rime TTS in real time over WebSockets for voice agents: choose between /ws3 (recommended JSON) and /ws (raw binary), with word-level timestamps and interruption handling.

tl;dr: Use /ws3.

/ws3 is Rime’s recommended WebSocket endpoint. It serves every current Rime TTS model, delivers the lowest possible TTFB via direct model streaming, and returns audio as structured JSON messages. Unless you have a specific reason to use a legacy endpoint, start here.

Authentication

Both WebSocket endpoints (/ws, /ws3) use the same bearer token as the rest of the Rime API. Send Authorization: Bearer YOUR_API_KEY as a connection header when establishing the connection, and pass synthesis arguments (speaker, modelId, audioFormat, lang, …) as query parameters on the connection URL. See API authentication for how to create a key, and the per-endpoint reference pages below for complete, runnable connection code.

Send the token in the Authorization header; connections without valid credentials are rejected with 401.

Your API key is a secret: keep it out of client-side code. The browser WebSocket API can’t set request headers, so connect from a server-side process (Node, Python, etc.), or proxy the connection through your own backend. The voice-agent guides show a complete browser ↔ server ↔ Rime bridge.

Choosing an endpoint

Rime offers two WebSocket endpoints.

	/ws3	/ws
Message format	JSON	Raw binary audio
Coda	✅	✅
Mist v3	✅	✅
Word-level timestamps	✅	❌
Context IDs	✅	❌
TTFB optimization	✅ Actively optimized	❌

---

/ws3: JSON WebSocket (recommended)

wss://users-ws.rime.ai/ws3

This is the endpoint Rime actively invests in. It serves every current Rime TTS model and streams audio as base64-encoded JSON chunks. TTFB is minimized by streaming model responses directly from the engine as they are produced.

Supported models:

• coda
• mistv3

If you’re onboarding to Rime’s WebSocket API for the first time, use /ws3. It’s the more capable of the two endpoints.

---

/ws: Binary WebSocket (legacy)

wss://users-ws.rime.ai/ws

/ws sends and receives raw audio bytes rather than JSON. It does not benefit from TTFB optimization. This endpoint is suited for clients that need raw PCM/binary audio and cannot handle JSON framing.

Supported models:

• coda
• mistv3

/ws returns raw audio bytes with no JSON framing. It does not support word-level timestamps, context IDs, or structured error events. If you need any of these features, use /ws3 instead.

---

Features available on /ws3

Word-level timestamps

/ws3 returns word-level timing alongside the audio. Rime estimates the timings from the text rather than measuring the audio. This is useful for tracking which words have already been spoken, for example, when an end-user interrupts the assistant mid-sentence.

type TimestampsEvent = {
  type: "timestamps",
  word_timestamps: {
    words: string[],
    start: number[],
    end: number[],
  },
  contextId: string | null,
}

Context IDs

On /ws3, you can attach a contextId to any text message and it will be echoed back on the corresponding audio chunk event. This is useful for correlating audio output to specific turns or requests in a multi-turn conversation.

{
  "text": "Hello, how can I help you today?",
  "contextId": "turn-001"
}

Rime does not maintain multiple simultaneous context IDs. The audio chunk event will carry the most recent context ID that was active at the time the audio was requested. If you send two messages before any audio is synthesized, only the later context ID will be reflected on the first audio chunk. Once set, a context ID also persists across subsequent messages that don’t provide one; events keep carrying the previous ID until you send a new one.

---

Operations

In addition to sending text, your client can send structured operation messages to control the synthesis pipeline.

flush

Synthesizes the current text buffer immediately and sends the resulting audio.

{ "operation": "flush" }

Use flush explicitly in segment=never mode. See the Segmentation guide for details.

clear

Discards the accumulated text buffer without synthesizing it. Useful when the user interrupts the assistant and you want to cancel queued speech.

{ "operation": "clear" }

eos (end of stream)

Synthesizes whatever remains in the buffer, sends a done event for that audio, then immediately closes the connection. If the buffer is already empty, the server closes without a done event.

{ "operation": "eos" }

---

Synthesis completion (done)

After all audio for a synthesis batch has been sent, the server emits a done event. This is the signal that the current utterance is fully delivered. If the client triggers further synthesis, another done will follow.

type DoneEvent = {
  type: "done",
  contextId: string | null,
}

done fires at different points depending on the segment setting:

• segment=never: fires once per flush, after all audio for that flush has been sent. eos also fires done for any content remaining in the buffer.
• segment=bySentence / segment=immediate: fires once per synthesis run, after the last segment completes and the buffer is empty. Intermediate sentence boundaries do not emit done.
• eos (all modes): emits done for any remaining buffered content before closing the connection. If the buffer is empty, the server closes without a done; treat socket close as terminal, not just done.

See Segmentation and behavior settings for details on each mode.

---

Reference & code examples

Each endpoint has a reference page with full message schemas and runnable connection code (Python websockets):

Endpoint	Coda	Mist v3
/ws3 (JSON)	Coda	Mist v3
/ws (binary)	Coda	Mist v3

Control synthesis timing

The segment parameter controls how text is buffered and when synthesis begins.

Segmentation & behavior settings

Compare segment=never, segment=bySentence, and segment=immediate.