Quickstart

Deploy a licensed Rime API and TTS service pair on an NVIDIA host with Docker Compose.

Run Rime on your own NVIDIA host as a paired API service and TTS model service. Each model deployment needs one of each service; multiple pairs can share a machine when capacity allows.

On-prem deployment requires registry access, a Rime license, and API credentials. Contact [e-mail] before starting. The running services also need outbound HTTPS access to Rime’s usage and license endpoints.

Architecture

The API service handles HTTP and WebSocket requests and verifies the license. The TTS service performs model inference.

On-prem keeps all audio and text inside your own network: no audio or text reaches Rime’s cloud. That supports data-residency requirements and compliance with data privacy and protection regulations.

Published latency figures

• Coda: This page documents the current image and service-pair setup. Rime has not published Coda on-prem performance numbers.
• Mist v2: Rime measured median latency of 175ms with randomly generated sentences between 40 and 50 characters on A10Gs and similar GPUs.

Prerequisites

Hardware requirements

• GPU
  • For Mist
    • NVIDIA T4, L4, A10, or higher
  • For Coda
    • Confirm GPU requirements with Rime before provisioning
• Storage
  • 50 GB storage
• CPU
  • 8 vCPUs
• Memory requirements
  • 32 GiB

Software requirements

• Supported Linux Distributions
  • Debian 12 (bookworm), x86_64
  • Ubuntu Server 24.04 (noble), x86_64
• NVIDIA drivers
  • Minimum: 580.65.06 (R580) for the default native CUDA mode
  • R535 or R570 hosts: supported only with RIME_CUDA_COMPAT_MODE=compat on forward-compatibility-eligible GPUs
  • Recommended: 595.71.05 or newer
• Docker
• NVIDIA Container Toolkit

Installations

NVIDIA drivers

Follow https://www.nvidia.com/en-us/drivers to install the latest NVIDIA drivers, or use the following instructions on Debian-based systems:

NVIDIA Driver Installation (Debian-based)

# Update packages
sudo apt-get update

# Install basic toolchain and kernel headers
sudo apt-get install -y gcc make wget linux-headers-$(uname -r)

# Download and install the NVIDIA driver.
NVIDIA_DRIVER_VERSION=580.95.05
NVIDIA_DRIVER_PATH=/opt/NVIDIA-Linux-x86_64-${NVIDIA_DRIVER_VERSION}.run
sudo rm -f "${NVIDIA_DRIVER_PATH}"
sudo wget "https://us.download.nvidia.com/tesla/${NVIDIA_DRIVER_VERSION}/NVIDIA-Linux-x86_64-${NVIDIA_DRIVER_VERSION}.run" -O "${NVIDIA_DRIVER_PATH}"
sudo chmod +x "${NVIDIA_DRIVER_PATH}"
sudo "${NVIDIA_DRIVER_PATH}" --silent --no-questions

Docker

Follow https://docs.docker.com/engine/install to install Docker on your system.

Optionally, add the current user to the docker group for convenience: https://docs.docker.com/engine/install/linux-postinstall. The code snippets below assume that you can run docker as the current login.

NVIDIA Container Toolkit

Follow https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html to install the NVIDIA Container Toolkit.

Follow both the Installation and the Configuration sections.

Verification

To verify that you have all the prerequisites installed, run the following command:

Verify Prerequisites

docker run --rm --gpus all nvidia/cuda:13.2.0-base-ubi9 nvidia-smi

You should see your GPU listed in the output, alongside the driver version and CUDA version.

Firewall requirements

The Rime API service listens on port 8000 for HTTP, port 8002 for binary WebSockets, and port 8003 for JSON WebSockets.

Allow the following outbound traffic in your firewall rules:

• https://optimize.rime.ai/usage: the API service calls this endpoint to register on-prem usage with Rime’s usage service.
• https://optimize.rime.ai/license: the API service calls this endpoint to verify that your on-prem license is active.
• us-docker.pkg.dev on port 443: container image registry.

API and registry credentials

Generate an API key

In the Rime dashboard, create an API key on the API Tokens page. Rime supplies the Artifact Registry key and license access separately.

Deployment

Select the model and API images, configure the service pair, then start it with Docker Compose.

Artifact Registry login

Use the Artifact Registry key provided by Rime:

Log in to Artifact Registry

cat KEY-FILE | docker login -u _json_key --password-stdin https://us-docker.pkg.dev

Container images

Image tags

All model images and the API image are published together under a single YYYYMMDD tag. Deploy the same tag across every container; mixing tags pairs components that were never released or tested together.

The current release tag is 20260801.

Substitute that tag for <tag> in the image references below, and update all containers together when you upgrade.

TTS service

Coda (multilingual)

The Coda v1 images can be found at us-docker.pkg.dev/rime-labs/coda/v1/coda:<tag>.

• The supported languages are: ar, de, en, es, fr, it, ja, pt.

Hindi is registered for Coda on-prem but ships in images tagged after 2026-08-04; it is not in 20260801.

Mist v3 (multilingual)

The Mist v3 images can be found at us-docker.pkg.dev/rime-labs/mist/v3/omni:<tag>

• The supported languages are: de, en, es, es-mx, fr. The image also bundles the English clones backend, which cannot be selected directly.

API service

• us-docker.pkg.dev/rime-labs/api/service:<tag>

Use the same YYYYMMDD tag as the model images.

Docker Compose configuration

Create a compose.yml file that defines the services and their configurations:

compose.yml

version: '3.8'
services:
  api:
    image: us-docker.pkg.dev/rime-labs/api/service:<tag>
    depends_on:
      - model
    ports:
      - "8000:8000"
      - "8002:8002" # binary websockets api
      - "8003:8003" # json websockets api
    restart: unless-stopped
    environment:
      - MODEL_URL=http://model:8080/invocations

  model:
    image: us-docker.pkg.dev/rime-labs/<model>/<version>/<variety>:<tag>
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
              count: all
    restart: unless-stopped

When running on Kubernetes, ensure that MODEL_URL points to http://0.0.0.0:8080/invocations instead of the Docker Compose service name.

Authentication configuration

By default, callers must pass their Rime API key in every request via the Authorization: Bearer <key> header. Two additional environment variables let you configure authentication at the deployment level instead.

Pre-configuring the API key (RIME_API_KEY)

If you set RIME_API_KEY, the API service uses it to authenticate with the Rime license server for HTTP requests on port 8000, and HTTP callers do not need to include an API key. WebSocket clients on ports 8001, 8002, and 8003 must still pass an API key on the connection request; the service caches the resulting license for subsequent connections.

You can supply it as an environment variable:

compose.yml

environment:
  - MODEL_URL=http://model:8080/invocations
  - RIME_API_KEY=<your-rime-api-key>

Or mount it as a secret file at /secrets/rime_api_key inside the container:

compose.yml

services:
  api:
    image: us-docker.pkg.dev/rime-labs/api/service:<tag>
    environment:
      - MODEL_URL=http://model:8080/invocations
    volumes:
      - /run/secrets/rime_api_key:/secrets/rime_api_key:ro

If you set neither, callers keep passing their API key in the per-request Authorization header.

Alternate API key header (API_KEY_HEADER)

On platforms that intercept the Authorization header, set API_KEY_HEADER to the name of an alternate header that callers will use to pass their Rime API key:

compose.yml

environment:
  - MODEL_URL=http://model:8080/invocations
  - API_KEY_HEADER=x-my-platform-rime-api-key

Callers then authenticate with:

curl -H "x-my-platform-rime-api-key: <your-rime-api-key>" ...

Platform API key (PLATFORM_API_KEY)

On platforms that require authenticated inter-container requests, set PLATFORM_API_KEY so the API service can reach the model backend. You can also mount it as a secret at /secrets/platform_api_key:

compose.yml

environment:
  - MODEL_URL=http://model:8080/invocations
  - PLATFORM_API_KEY=<your-platform-api-key>

Start Docker Compose

Start Docker Compose

docker compose up -d

Allow approximately five minutes for model warm-up before sending the first synthesis request.

Verify health

Health check

curl http://localhost:8000/health

A ready deployment returns HTTP 200 with apiStatus: "ok" and modelReachable: true. licenseStatus reads expired-or-not-set until the first authenticated request initializes the license, then valid; it does not affect the status code:

{
  "apiStatus": "ok",
  "timestamp": "2026-07-24T18:00:00.000Z",
  "licenseStatus": "valid",
  "modelReachable": true
}

AWS video walkthrough

Requests and response formats

HTTP requests

Request:

Request example

curl -X POST "http://localhost:8000" \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "text": "I would love to have a conversation with you. The new model is out.",
  "speaker": "joy",
  "modelId": "mist"
}' -o result_mist.txt

Response:

Response format

{"audioContent":{"model_output":"<base64>"}}

Sample response file: result.txt

Receiving a response in MP3 format

Request:

Request example

curl -X POST "http://localhost:8000" -H "Authorization: Bearer <API KEY>" -H "Content-Type: application/json" -H "Accept: audio/mpeg" -d '{
  "text": "I would love to have a conversation with you.",
  "speaker": "joy",
  "modelId": "mist"
}' -o result.mp3

Response:

Sample response file: result.mp3

Receiving a response in PCM (raw) format

Request:

Request example

curl -X POST "http://localhost:8000" -H "Authorization: Bearer <API KEY>" -H "Content-Type: application/json" -H "Accept: audio/L16" -d '{
  "text": "I would love to have a conversation with you.",
  "speaker": "joy",
  "modelId": "mist"
}' -o result.pcm

Response:

Sample response file: result.pcm

WebSocket endpoints

JSON websockets

The API service serves the JSON WebSocket endpoint for Coda and Mist models on port 8003. For example, ws://localhost:8003 is equivalent to Rime’s cloud JSON WebSocket API.

Spanish is not accepted on the on-prem JSON WebSocket ports. Sending lang=spa to port 8003 or 8001 returns HTTP 400, including for Coda deployments that serve Spanish voices. This restriction does not exist on the cloud endpoint. For Spanish on-prem, use the binary WebSocket endpoint or the HTTP endpoint, which lose word timestamps and context IDs.

See the Coda JSON WebSocket docs or Mist JSON WebSocket docs, depending on the model backend you configured.

Non-JSON websockets

The API service serves the non-JSON WebSocket endpoint on port 8002. For example, ws://localhost:8002 is equivalent to Rime’s cloud WebSocket API.

Deprecated Mist endpoint (port 8001)

The API service also serves a deprecated WebSocket endpoint on port 8001. It is compatible only with the Mist model family. Use the current WebSocket endpoints for new integrations.