Quickstart

Run a licensed Coda or Mist v3 engine on your NVIDIA host with Docker Compose.

One model container serves HTTP, gRPC, and WebSocket synthesis. It needs a mounted runtime license and metering API key to start, and it doesn’t authenticate callers, so your gateway controls who can use it.

Before you start, ask [e-mail] for registry access and a runtime license that matches your selected image. If you already run an API/model pair, follow Migrate an existing deployment.

Architecture

The engine checks the license locally at startup and sends aggregate usage counts to Rime over HTTPS. Your synthesis text and audio stay within your deployment.

A runtime license authorizes the engine to run. It does not authenticate callers in this mode. The metering API key only authenticates outbound usage reports. Keep both engine ports private. Your gateway must authenticate callers and provide TLS. The Compose example binds both ports to the host’s loopback interface.

Prerequisites

Hardware requirements

• GPU
  • For Mist
    • NVIDIA T4, L4, A10, or higher
  • For Coda
    • Confirm GPU requirements with Rime before provisioning
• Storage
  • 50 GB storage
• CPU
  • 8 vCPUs
• Memory requirements
  • 32 GiB

Software requirements

• Supported Linux Distributions
  • Debian 12 (bookworm), x86_64
  • Ubuntu Server 24.04 (noble), x86_64
• NVIDIA drivers
  • Minimum: 580.65.06 (R580) for the default native CUDA mode
  • R535 or R570 hosts: supported only with RIME_CUDA_COMPAT_MODE=compat on forward-compatibility-eligible GPUs
  • Recommended: 595.71.05 or newer
• Docker
• NVIDIA Container Toolkit

Installations

NVIDIA drivers

Follow https://www.nvidia.com/en-us/drivers to install the latest NVIDIA drivers, or use the following instructions on Debian-based systems:

NVIDIA Driver Installation (Debian-based)

# Update packages
sudo apt-get update

# Install basic toolchain and kernel headers
sudo apt-get install -y gcc make wget linux-headers-$(uname -r)

# Download and install the NVIDIA driver.
NVIDIA_DRIVER_VERSION=580.95.05
NVIDIA_DRIVER_PATH=/opt/NVIDIA-Linux-x86_64-${NVIDIA_DRIVER_VERSION}.run
sudo rm -f "${NVIDIA_DRIVER_PATH}"
sudo wget "https://us.download.nvidia.com/tesla/${NVIDIA_DRIVER_VERSION}/NVIDIA-Linux-x86_64-${NVIDIA_DRIVER_VERSION}.run" -O "${NVIDIA_DRIVER_PATH}"
sudo chmod +x "${NVIDIA_DRIVER_PATH}"
sudo "${NVIDIA_DRIVER_PATH}" --silent --no-questions

Docker

Follow https://docs.docker.com/engine/install to install Docker on your system.

Optionally, add the current user to the docker group for convenience: https://docs.docker.com/engine/install/linux-postinstall. The code snippets below assume that you can run docker as the current login.

NVIDIA Container Toolkit

Follow https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html to install the NVIDIA Container Toolkit.

Follow both the Installation and the Configuration sections.

Verification

This command runs nvidia-smi inside a CUDA container, so it checks the driver, Docker, and the NVIDIA Container Toolkit together:

Verify Prerequisites

docker run --rm --gpus all nvidia/cuda:13.2.0-base-ubi9 nvidia-smi

The output should list your GPU, the driver version, and the CUDA version. If the command fails, the engine container won’t get GPU access either, so fix it before you continue.

Firewall requirements

Connection	Port or endpoint	Purpose
Your application or gateway to the engine	TCP 8080	HTTP synthesis, health checks, and WebSocket /ws
Your gRPC client or gateway to the engine	TCP 7463	gRPC synthesis and health checks
Engine to Rime	HTTPS https://optimize.rime.ai/metering	Aggregate usage reports
Deployment host to Artifact Registry	HTTPS us-docker.pkg.dev	Image pulls

If the engine cannot reach /metering, synthesis continues. The engine keeps reports in memory and retries them. Its queue holds up to 168 reporting windows, each one hour by default. Adding a window beyond that limit drops the oldest pending window. A process exit can also lose pending counts.

The licensed engine sends reports to /metering only. It does not call /license or /usage.

API and registry credentials

Rime supplies registry access and the signed runtime license. In the dashboard for the billed account, open API Tokens and create the metering API key.

Store the license as ./secrets/license and the API key as ./secrets/api-key on the deployment host. The engine must be able to read both files inside the container. Keep them outside source control.

Log in to Artifact Registry with the key Rime supplied:

cat KEY-FILE | docker login -u _json_key --password-stdin https://us-docker.pkg.dev

Image tags

Use the release tag in the .env example below for Coda or Mist v3. Confirm the selected image and runtime license with Rime for standalone licensed deployment.

TTS service

Model	Image repository
Coda v1	us-docker.pkg.dev/rime-labs/coda/v1/coda
Mist v3	us-docker.pkg.dev/rime-labs/mist/v3/omni

Create .env next to compose.yml to select the Coda image at the current release tag:

RIME_MODEL_IMAGE=us-docker.pkg.dev/rime-labs/coda/v1/coda:20261006b

For Mist v3, replace the repository path with us-docker.pkg.dev/rime-labs/mist/v3/omni and keep the same tag.

On R535 or R570 hosts with forward-compatibility-eligible GPUs, add RIME_CUDA_COMPAT_MODE=compat to .env. Compose uses native when this setting is absent.

Keep the image’s command and entrypoint. The image already contains the model files, pipeline configuration, license verifier keys, and package identity. You do not need an API service image in this mode.

Use Models and Voices to choose a model and voice. Confirm the languages in your selected image with Rime.

Docker Compose configuration

Create compose.yml to run the model with GPU access and read-only credential mounts:

services:
  model:
    image: ${RIME_MODEL_IMAGE:?Set RIME_MODEL_IMAGE in .env}
    environment:
      RIME_CUDA_COMPAT_MODE: ${RIME_CUDA_COMPAT_MODE:-native}
    ports:
      - "127.0.0.1:8080:8080"
      - "127.0.0.1:7463:7463"
    volumes:
      - ./secrets/license:/var/run/secrets/rime/license:ro
      - ./secrets/api-key:/var/run/secrets/rime/api-key:ro
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
              count: all
    stop_grace_period: 11m
    restart: unless-stopped

Publish only the ports your clients need. If your gateway shares the engine’s private Docker network, it can use model:8080 and model:7463 without published host ports. On Kubernetes, mount the same secrets and route a private Service to these container ports.

Mounted files keep credentials out of the container environment. If your platform mounts secrets elsewhere, set RIME_LICENSE_FILE and RIME_API_KEY_FILE to those paths.

You can also set RIME_LICENSE to the base64 license text Rime supplied and RIME_API_KEY to the key value. Do not encode the license again. Restart the engine after you change either credential.

Startup fails if a configured license is missing, invalid, or expired, or if the metering API key is missing. With no license source at all, the engine starts in legacy request-license mode. You must supply a runtime license to use standalone licensed mode.

Allow time for shutdown. The eleven-minute stop grace period covers the default ten-minute request drain plus cleanup. See Load balancing before you change it.

Start and verify

Start the deployment, then wait for the engine to load and warm up. This readiness check retries for up to ten minutes, with a five-second timeout per attempt:

docker compose up -d
curl --fail --retry 120 --retry-delay 5 --retry-connrefused \
  --retry-max-time 600 --max-time 5 http://localhost:8080/readyz

HTTP 200 with ok means the engine is ready. During loading or drain, it returns HTTP 503. If the retries fail, check docker compose logs model before you send synthesis requests. See Health checks and metrics for liveness and gRPC checks.

Send this request to write output.wav. Coda and Mist v3 both support the example voice, luna. Use a voice available in your image:

curl --fail-with-body http://localhost:8080/ \
  -H 'Content-Type: application/json' \
  -H 'Accept: audio/wav' \
  -d '{"text":"Hello from Rime.","speaker":"luna","language":"en"}' \
  -o output.wav

Play output.wav. If you hear the sentence, the license, GPU access, and engine are working. If curl reports an HTTP error, output.wav holds the engine’s error response, not audio. Engine protocols covers the request fields, response handling, and gRPC and WebSocket examples.

Migrate an existing deployment

Keep the previous API/model pair available while you test a separate standalone deployment. Verify each protocol your application uses before you switch traffic.

Previous API/model pair	Standalone licensed engine
API service plus model service	One model container with license and metering key mounts
API service calls /license and /usage	Local license verification; engine calls /metering
HTTP on API port 8000	HTTP on engine port 8080; clients consume audio bytes
WebSocket ports 8001, 8002, and 8003	/ws on port 8080, with a different message protocol
API service checks caller API keys	Your gateway checks callers for all three protocols
API /health	Engine /livez, /readyz, and gRPC health service

Remove MODEL_URL, API_KEY_HEADER, and PLATFORM_API_KEY from the new deployment. They configure the old API service. Mount the metering key on the engine, then update your proxy routes, firewall rules, probes, and clients. The engine image selects the model. A request’s modelId cannot route it to another model.

Changing a WebSocket URL is not enough to migrate a client. An old /ws3 client must also send the messages that /ws expects. Follow the WebSocket migration rules and check audio completion, cancellation, and errors before you switch traffic.

Existing API/model pairs keep their request-license and API-side usage reporting behavior when no offline runtime license is mounted.