> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Engine protocols

> Call a standalone licensed Coda or Mist engine over HTTP, gRPC, or WebSocket.

A standalone engine serves the model in its image over HTTP, gRPC, and WebSocket. The image selects the model, so a request can't switch to another one. These interfaces apply to the [standalone licensed deployment](https://docs.rime.ai/docs/on-prem/quickstart.md). The cloud API serves the same `rime.TextToSpeech` gRPC service, as [Hosted gRPC](#hosted-grpc) describes. The legacy API container has its own contract.

| Protocol | Packaged engine address | Input | Output |
| :- | :- | :- | :- |
| HTTP | `http://localhost:8080/` | One JSON request with complete text | Streamed audio bytes |
| gRPC | `localhost:7463`, service `rime.TextToSpeech` | Complete text or a stream of text chunks | Protobuf audio messages |
| WebSocket | `ws://localhost:8080/ws` | One-shot or streaming contexts | JSON or protobuf event envelopes |

The model image configures the ports above. Use those ports for the quickstart. A bare engine binary defaults to HTTP port `7464` and gRPC port `7463`.

## Caller authentication

Your gateway must authenticate callers and provide TLS for every exposed protocol. In offline-license mode, the engine authorizes synthesis through the mounted runtime license. HTTP and gRPC callers do not need to send the metering key or license with each request.

> [!WARNING]
>
> In offline-license mode, the engine accepts WebSocket upgrades without credentials, so it provides no access control of its own. The WebSocket `config` message carries synthesis defaults only. If it sets `authorization` or `license`, the engine sends an `invalid_input` error and closes the connection. Keep the metering API key on the engine.

## HTTP

Send the complete text to `POST /`. You can also use its alias, `POST /invocations`. Set `Content-Type: application/json` and choose an audio format with `Accept`, such as `audio/wav` or `audio/mpeg`.

| JSON field | Meaning |
| :- | :- |
| `text` | Required text to synthesize |
| `speaker` | Voice supported by the image, for example `luna` |
| `language` | BCP-47 language tag, for example `en` (`lang` is an alias) |
| `samplingRate` | Requested output sample rate in Hz, subject to model support |
| `timeScaleFactor` | Duration multiplier, where `1` preserves the generated duration |
| `splitStrategy` | `sentence` or `none`. Incremental input is available through gRPC or WebSocket |

Check the response `Content-Type` before you decode the audio.

> [!NOTE]
>
> For compatibility with existing HTTP clients, the engine also accepts `audioFormat` in the request body. Valid short values, such as `wav` or `mp3`, override `Accept`. If the field is absent or invalid, the engine uses `Accept`. Without either a valid body format or an `Accept` header, the request fails. Use `Accept` for new clients.

This request writes the WAV response to `output.wav`:

```bash theme={null}
curl --fail-with-body http://localhost:8080/ \
  -H 'Content-Type: application/json' \
  -H 'Accept: audio/wav' \
  -d '{"text":"Hello from Rime.","speaker":"luna","language":"en"}' \
  -o output.wav
```

Read the response as audio bytes. The engine does not use the legacy `audioContent.model_output` JSON wrapper. Before audio starts, a failure can return an HTTP error. After audio starts, a failure aborts the response body. HTTP 200 alone is not proof that synthesis completed.

Word timestamps are currently unavailable over all three standalone engine protocols.

## Python and JavaScript packages

Rime publishes Python [`rime-api`](https://pypi.org/project/rime-api/) and JavaScript [`@rimelabs/api`](https://www.npmjs.com/package/@rimelabs/api) with generated TTS message definitions for the engine's gRPC and WebSocket protocols. Both packages include the Protobuf and AsyncAPI schemas. Version 0.5.0 and newer also include gRPC clients. In Python, install `rime-api[grpc]` and import `TextToSpeechStub` from `rime_api.text_to_speech_pb2_grpc`. In Node.js, install `@grpc/grpc-js` and import `TextToSpeechClient` from `@rimelabs/api/grpc`. Applications supply the engine address, TLS settings, authentication metadata, and audio handling.

Python requires version 3.10 or newer. Install the package and gRPC runtime for your application language:

```shell Python theme={null}
uv add 'rime-api[grpc]>=0.5.0'
```

```shell JavaScript / TypeScript theme={null}
npm install '@rimelabs/api@>=0.5.0' '@grpc/grpc-js@^1.14.0' '@bufbuild/protobuf@^2.11.0'
```

If you only use message definitions for WebSockets, omit the Python `grpc` extra or the Node.js `@grpc/grpc-js` dependency. Your application supplies connection settings, authentication metadata, cancellation policy, and audio playback. See the [PyPI package description](https://pypi.org/project/rime-api/) for imports and JSON or binary encoding examples. The [cloud WebSocket endpoints](https://docs.rime.ai/docs/websockets.md) use different message formats from these engine envelopes.

## gRPC

The engine exposes `rime.TextToSpeech` without gRPC reflection. Packaged clients do not need reflection. Supply the schema when you use `grpcurl`. Choose a package release compatible with your engine image:

* Python wheel: `rime_api/schema/rime/text_to_speech.proto`.
* npm package: `schema/rime/text_to_speech.proto`.

The schema directory also contains `google/protobuf/duration.proto` and `google/rpc/status.proto`, including their imports, for tools such as `grpcurl`.

| RPC | Behavior |
| :- | :- |
| `Synthesize` | Accepts one `SynthesisRequest` and streams `SynthesisResponseStream` messages |
| `SynthesizeStreaming` | Accepts a stream of `StreamingSynthesisRequest` and streams the same response messages |
| `NormalizeText` | Returns `normalizedSentences` |
| `GetSupportedLanguages` | Returns the image's language tags |
| `GetSupportedSpeakers` | Lists Coda speakers (Mist returns `UNIMPLEMENTED`) |

With [grpcurl](https://github.com/fullstorydev/grpcurl) installed, this command sends complete text and prints the streamed response messages. Replace the example import path `./rime-protos` with your schema directory:

```bash theme={null}
grpcurl -plaintext \
  -import-path ./rime-protos \
  -proto rime/text_to_speech.proto \
  -d '{"text":"Hello from Rime.","speaker":"luna","language":"en","audioParameters":{"audioFormat":"audio/wav"}}' \
  localhost:7463 rime.TextToSpeech/Synthesize
```

Keep `-plaintext` for the private local listener. Use TLS through your gateway. grpcurl prints JSON with base64 `audio` fields. A generated protobuf client receives those fields as bytes. Append the audio payloads in order to build the audio file. Writing whole protobuf messages would include data outside the audio payload.

For incremental input, send one `header` followed by `textChunk` messages. Close the request side when you have sent all the text, then keep reading until the response stream completes. This example sends two complete sentences:

```bash theme={null}
grpcurl -plaintext \
  -import-path ./rime-protos \
  -proto rime/text_to_speech.proto -d @ \
  localhost:7463 rime.TextToSpeech/SynthesizeStreaming <<'JSON'
{"header":{"speaker":"luna","language":"en","audioParameters":{"audioFormat":"audio/wav"}}}
{"textChunk":"Hello from Rime. "}
{"textChunk":"This is the second sentence."}
JSON
```

If your input arrives a token at a time, set `header.splitStrategy` to `SPLIT_STRATEGY_ACCUMULATE_SENTENCE`. The engine collects text into sentences before synthesis. The selected image still determines model-specific synthesis behavior.

To stop a streaming synthesis early, send `{"stop":{}}` on the request stream at any point after the `header`. The engine ends the response with the audio it has made, and the RPC finishes with OK. A half-closed request side can no longer carry `stop`, so if you might stop after the text is complete, send `{"end":{}}` to end the text and keep the request side open. The engine ignores a `textChunk` after `end`.

Check the final RPC status. Anything other than OK means synthesis is incomplete. Cancelling the RPC also stops generation. The schema defines timestamp fields and a response trailer, but the standalone engine currently sends only audio messages. Setting `timestamps.enable` has no effect.

### Hosted gRPC

Rime's cloud API serves the same `rime.TextToSpeech` service over TLS on port 443. Call `coda.api.rime.ai:443` for Coda or `mist.api.rime.ai:443` for Mist v3. Send your API key as `authorization: Bearer` metadata on each call. This command reads the key from `RIME_API_KEY` and sends the complete-text example to Coda:

```bash theme={null}
grpcurl \
  -H "authorization: Bearer $RIME_API_KEY" \
  -import-path ./rime-protos \
  -proto rime/text_to_speech.proto \
  -d '{"text":"Hello from Rime.","speaker":"luna","language":"en","audioParameters":{"audioFormat":"audio/wav"}}' \
  coda.api.rime.ai:443 rime.TextToSpeech/Synthesize
```

## WebSocket

Open `/ws` on the engine's HTTP port. Offer `rime.v1.json` for JSON text responses or `rime.v1.binary` for protobuf binary responses. The engine defaults to JSON if you omit the subprotocol. Both encodings use `WebSocketRequest` and `WebSocketResponse` from the gRPC schema bundle.

Each frame contains one message envelope. JSON uses camelCase fields and base64 `audio` values. Binary frames contain protobuf envelopes. With either encoding, extract the `audio` payload before you write it to an audio file.

| Client message | Action |
| :- | :- |
| `config` | Set synthesis defaults once, before the first `start` |
| `start` with non-empty `text` | Synthesize the complete text, with no `end` needed |
| `start` with empty `text` | Open a streaming context |
| `text` | Append input to that streaming context |
| `end: {}` | Finish input, then continue reading until `done` |
| `stop: {}` | Stop a streaming context's synthesis early, then continue reading until `done` |
| `cancel: {}` | Cancel the context and expect `cancelled` |

Wait for `ready` before you send `start`. For each context, the server sends `started`, zero or more `audio` messages, then `done`, `cancelled`, or `error`. Match responses by `contextId`. An error with an empty context ID applies to the connection. You can keep the connection open for later contexts.

Save this example as `stream.mjs` and run it with Node.js 22 or later against a private engine with a verified offline license. It sends no credentials. Running `node stream.mjs` writes `output.wav` only after the engine sends `done`:

```javascript theme={null}
import { writeFileSync } from "node:fs";

const socket = new WebSocket("ws://localhost:8080/ws", "rime.v1.json");
const chunks = [];
let completed = false;
let errorReported = false;
const send = (message) => socket.send(JSON.stringify(message));

socket.addEventListener("message", ({ data }) => {
  const message = JSON.parse(data);
  if (message.error) {
    console.error(message.error);
    errorReported = true;
    process.exitCode = 1;
    socket.close();
    return;
  }
  if (message.ready) {
    send({ contextId: "example", start: {
      text: "", speaker: "luna", language: "en",
      audioParameters: { audioFormat: "audio/wav" },
    } });
    send({ contextId: "example", text: "Hello from Rime. " });
    send({ contextId: "example", text: "This is the second sentence." });
    send({ contextId: "example", end: {} });
  }
  if (message.contextId !== "example") return;
  if (message.audio) chunks.push(Buffer.from(message.audio, "base64"));
  if (message.done) {
    writeFileSync("output.wav", Buffer.concat(chunks));
    completed = true;
    socket.close();
  }
  if (message.cancelled) socket.close();
});

socket.addEventListener("error", () => { process.exitCode = 1; });
socket.addEventListener("close", () => {
  if (!completed) {
    if (!errorReported) console.error("Synthesis ended before done");
    process.exitCode = 1;
  }
});
```

If your input arrives a token at a time, set `start.splitStrategy` to `SPLIT_STRATEGY_ACCUMULATE_SENTENCE`. Wait for `done` to confirm that all audio has arrived. This event does not currently include timestamp spans.

Migration from the API container's WebSocket ports requires changes to your client's messages. Move synthesis parameters from the old query parameters into `start`. Replace `flush`, `clear`, and `eos` with the context lifecycle above. Use `end` to finish input and `cancel` to stop a context. Decode the `audio` payload from each envelope.

For language support, read the languages in `ready` and test the voices in your image. The old API container's port-specific restrictions do not define this protocol.

## Routing, health, and failures

Route traffic using [HTTP or gRPC health checks](https://docs.rime.ai/docs/on-prem/metrics.md). Your gateway must support HTTP/2 gRPC and WebSocket upgrades. Set its timeouts to allow streams to stay open for the duration your application needs.

All three protocols share one inference capacity limit. At that limit, or during request drain, the engine refuses new synthesis. It returns gRPC `UNAVAILABLE`, HTTP 503, or a WebSocket `error` with kind `unavailable`.

Before you retry, check whether the client already received audio. Retry with backoff only if your application can avoid playing that audio again. See [Load balancing](https://docs.rime.ai/docs/on-prem/load-balancing.md) for stream routing and shutdown.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.