Legacy JSON WebSocket (/ws3)

Coda JSON WebSocket (/ws3): structured events with base64 audio chunks and word-level timestamps.

Python

import asyncio
import json
import websockets
import base64

class RimeClient:
    def __init__(self, speaker, api_key):
        self.url = f"wss://users-ws.rime.ai/ws3?speaker={speaker}&modelId=coda&audioFormat=mp3"
        self.auth_headers = {
            "Authorization": f"Bearer {api_key}"
        }
        self.audio_data = b''

    async def send_messages(self, websocket, messages):
        for message in messages:
            await websocket.send(json.dumps(message))

    async def handle_audio(self, websocket):
        while True:
            try:
                audio = await websocket.recv()
            except websockets.exceptions.ConnectionClosedOK:
                break
            message = json.loads(audio)

            if message['type'] == 'chunk':
              self.audio_data += base64.b64decode(message['data'])

            if message['type'] == 'timestamps':
                print("Rime model pronounced the words...\n")
                for w, t in zip(message['word_timestamps']['words'], message['word_timestamps']['start']):
                    print(f"'{w}' at time {t}")

    async def run(self, messages):
        async with websockets.connect(self.url, additional_headers=self.auth_headers) as websocket:
            await asyncio.gather(
                self.send_messages(websocket, messages),
                self.handle_audio(websocket),
            )

    def save_audio(self, file_path):
        with open(file_path, 'wb') as f:
            f.write(self.audio_data)
        print(f"\n Audio saved at {file_path}")


message = [
    {"text": "This "},
    {"text": "is "},
    {"text": "a "},
    {"text": "test "},
    {"operation":"clear"},
    {"text": "This "},
    {"text": "is "},
    {"text": "an "},
    {"text": "incomplete "},
    {"text": "sentence "},
    {"operation": "eos"},
]

client = RimeClient("astra", api_key="xxx")
asyncio.run(client.run(message))

client.save_audio("output.mp3")

The Rime API authenticates every request with a bearer token in the Authorization header: Authorization: Bearer YOUR_API_KEY. See API authentication for how to create a key.

This page describes the legacy /ws3 endpoint, which keeps working for existing integrations. For a new Coda integration, use Coda WebSocket v1.

Include modelId=coda in the connection query. If you omit it, the server routes the request to Mist v3 and speakers outside the Mist v3 catalog can fail with Speaker not found.

​

Overview

This endpoint sends and receives JSON events. All synthesis arguments are query parameters on the connection URL.

With the default segment=bySentence, the server buffers text until it ends a sentence (., ?, or !), so synthesis of the first message waits for a complete sentence. After the first utterance, audio playback usually covers that buffering.

​

Messages

​

Send

​

Text

Text for synthesis.

type TextMessage = {
  text: string,
  contextId?: string,
}

{
    "text": "this is the minimum text message."
}

{
    "text": "this is a text message with a context id.",
    "contextId": "159495B1-5C81-4C73-A51A-9CE10A08239E"
}

The server echoes contextId on the events for the audio it produces. It tracks one context ID at a time. Each synthesis request carries the most recent context ID at the moment the server issues that request, and the ID persists until you send a new one. A clear sent while no synthesis is in progress resets it to null. In the example above, if the first sentence is already being synthesized when the second message arrives, its audio carries contextId: null and the second sentence’s audio carries the UUID.

​

Clear

Discards the buffered text. Use it when the user interrupts.

{ "operation": "clear" }

​

Flush

Synthesizes whatever text is buffered and sends the audio.

{ "operation": "flush" }

​

EOS

Synthesizes whatever text is buffered, then closes the connection.

{ "operation" : "eos" }

​

Receive

​

Chunk

Carries a piece of the synthesized audio.

type Base64String = string

type AudioChunkEvent = {
  type: "chunk",
  data: Base64String,
  contextId: string | null,
}

data is base64-encoded audio in the audioFormat set on the connection. With pcm, a chunk can end mid-sample, so keep whole samples. contextId is the context ID of the text that produced it, if any.

​

Timestamps

The server sends one timestamps event per synthesis request (one or more buffered sentences), right after that request’s first audio chunk. The API service estimates the times from the text, speaker, and timeScaleFactor. They are not aligned to the generated audio. Use them to map a playback position back to the last word heard when the user interrupts.

Timestamps are emitted only when lang is en/eng or es/spa, or when lang is omitted. Requests in any other language receive chunk and done events with no timestamps event and no error (this includes fr, de, ja, pt, ar, and hi). Do not block playback while waiting for a timestamps event.

type TimestampsEvent = {
  type: "timestamps",
  word_timestamps: {
    words: string[],
    start: number[],
    end: number[],
  },
  contextId: string | null,
}

The three arrays inside word_timestamps are the same length and index-aligned: for a given index i, words[i] is spoken from start[i] to end[i]. Times are in seconds, measured from the beginning of the audio for the current synthesis. If a context id was attached to the text that produced this audio, it is included on the event.

Example payload:

{
  "type": "timestamps",
  "word_timestamps": {
    "words": ["Hello", "from", "Coda", "over", "JSON", "websockets."],
    "start": [0, 0.36106, 0.54159, 0.72212, 0.90265, 1.08318],
    "end":   [0.36106, 0.54159, 0.72212, 0.90265, 1.08318, 2.88848]
  },
  "contextId": null
}

​

Done

After the last audio chunk for a synthesis batch has been sent, the server emits a done event. This signals that the current synthesis is fully complete. If the client sends more text and triggers further synthesis, another done will follow.

type DoneEvent = {
  type: "done",
  contextId: string | null,
}

When exactly done fires depends on the segment setting. See Segmentation and behavior settings for full details.

​

Error

The server answers a malformed or unexpected message with an error event. It does not close the connection and keeps accepting well-formed messages, so your client decides whether to close.

type ErrorEvent = {
  type: "error",
  message: string,
}

​

Variable parameters

speaker · string · required

Must be a coda voice from the Rime voice catalog.

text · string · required

The text you’d like spoken. Character limit per request is 1,000 via the API and in the dashboard UI.

modelId · string · default: "mistv3"

Set this to coda. It is not strictly required, but if you omit it the server defaults to the Mist v3 backend, and speakers outside the Mist v3 catalog fail with a “Speaker not found” error.

audioFormat · string

One of wav, mp3 (or mpeg), ogg (Opus in OGG), webm (Opus in WebM), pcm (or l16), or mulaw (or pcmu). Unrecognized values fall back to wav.

lang · string · default: "en"

If provided, the language must match the language spoken by the provided speaker. Pass a BCP 47 language tag:

Tag	Language
en	English
es	Spanish
fr	French
pt	Portuguese
de	German
ja	Japanese
ar	Arabic
hi	Hindi
it	Italian
nl	Dutch

The 3-letter ISO 639-2 codes that older integrations send (eng, spa, fra, por, ger, jpn, ara, hin, ita, nld) remain accepted, and will stay accepted.

See the voices documentation for which speakers support each language.

samplingRate · int · default: "24000"

The sampling rate in Hz. Any positive integer is accepted, and values above the model’s native rate are upsampled. Common choices are 8000 (telephony), 16000, 22050, 24000, and 44100.

timeScaleFactor · float · default: "1.0"

The time scaling factor. Accepted range is 0.4 to 2.5. Values outside this range are clamped without an error. Values above 1.0 slow the audio down, and values below 1.0 speed it up.

segment · string · default: "bySentence"

Controls how text is segmented for synthesis. Available options:

• “immediate” - Synthesizes text immediately without waiting for complete sentences
• “never” - Never segments the text, waits for explicit flush or EOS
• “bySentence” (default) - Waits for complete sentences before synthesis

Note: For backward compatibility, setting immediate=true in query params is equivalent to segment=immediate. If a null value is provided, it will default to “bySentence”.

Python

import asyncio
import json
import websockets
import base64

class RimeClient:
    def __init__(self, speaker, api_key):
        self.url = f"wss://users-ws.rime.ai/ws3?speaker={speaker}&modelId=coda&audioFormat=mp3"
        self.auth_headers = {
            "Authorization": f"Bearer {api_key}"
        }
        self.audio_data = b''

    async def send_messages(self, websocket, messages):
        for message in messages:
            await websocket.send(json.dumps(message))

    async def handle_audio(self, websocket):
        while True:
            try:
                audio = await websocket.recv()
            except websockets.exceptions.ConnectionClosedOK:
                break
            message = json.loads(audio)

            if message['type'] == 'chunk':
              self.audio_data += base64.b64decode(message['data'])

            if message['type'] == 'timestamps':
                print("Rime model pronounced the words...\n")
                for w, t in zip(message['word_timestamps']['words'], message['word_timestamps']['start']):
                    print(f"'{w}' at time {t}")

    async def run(self, messages):
        async with websockets.connect(self.url, additional_headers=self.auth_headers) as websocket:
            await asyncio.gather(
                self.send_messages(websocket, messages),
                self.handle_audio(websocket),
            )

    def save_audio(self, file_path):
        with open(file_path, 'wb') as f:
            f.write(self.audio_data)
        print(f"\n Audio saved at {file_path}")


message = [
    {"text": "This "},
    {"text": "is "},
    {"text": "a "},
    {"text": "test "},
    {"operation":"clear"},
    {"text": "This "},
    {"text": "is "},
    {"text": "an "},
    {"text": "incomplete "},
    {"text": "sentence "},
    {"operation": "eos"},
]

client = RimeClient("astra", api_key="xxx")
asyncio.run(client.run(message))

client.save_audio("output.mp3")