Build a voice agent (Express)

Build a minimal voice-agent loop on one Express server: the browser captures speech, your server decides what to say, and Rime speaks the reply. This starter uses browser speech recognition and a stub response function; replace both with production components. Keep RIME_API_KEY on the server and call Rime over HTTPS with fetch. No Rime-specific npm package is required; Express is the only dependency.

Browser (mic + playback)  →  POST /api/tts (Express)  →  POST https://users.rime.ai/v1/rime-tts

Prerequisites

A Rime API key from the API Tokens page, exported as RIME_API_KEY
Node.js 20.11+ and npm install express

1. Server: Express app with a TTS route

Create server.mjs:

server.mjs

import path from "node:path";
import express from "express";

const app = express();
app.use(express.json());

app.post("/api/tts", async (req, res) => {
  const { text, speaker = "astra", modelId = "coda" } = req.body ?? {};
  if (!text) return res.status(400).json({ error: "text is required" });

  const rimeRes = await fetch("https://users.rime.ai/v1/rime-tts", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.RIME_API_KEY}`,
      "Content-Type": "application/json",
      Accept: "audio/mpeg", // MP3; see the API reference for Opus, WAV, PCM, μ-law
    },
    body: JSON.stringify({ text, speaker, modelId }),
  });

  if (!rimeRes.ok) {
    return res
      .status(rimeRes.status)
      .json({ error: `Rime API error ${rimeRes.status}`, detail: await rimeRes.text() });
  }

  res.setHeader("Content-Type", "audio/mpeg");
  res.send(Buffer.from(await rimeRes.arrayBuffer()));
});

app.get("/", (_req, res) => {
  res.sendFile(path.join(import.meta.dirname, "index.html"));
});

app.listen(3000, () => console.log("Rime voice agent on http://localhost:3000"));

Verify it works before touching the frontend:

RIME_API_KEY=your_key node server.mjs &
curl -X POST http://localhost:3000/api/tts \
  -H 'Content-Type: application/json' \
  -d '{"text": "Hello from my Express voice agent."}' \
  --output test.mp3

test.mp3 should be playable audio. A 401 means RIME_API_KEY isn’t visible to the server process.

2. Client: mic in, Rime audio out

Create index.html next to server.mjs. It uses the browser’s built-in SpeechRecognition for input (Chrome/Edge/Safari), a stub respond() function as the agent brain, and your /api/tts route for the voice:

index.html

<!doctype html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>Rime voice agent</title>
    <style>
      main { max-width: 480px; margin: 4rem auto; font-family: sans-serif; }
    </style>
  </head>
  <body>
    <main>
      <h1>Rime voice agent</h1>
      <button id="speak">🎤 Speak</button>
      <ul id="messages"></ul>
      <audio id="player" hidden></audio>
    </main>
    <script>
      // Replace this stub with a call to your LLM of choice.
      function respond(userText) {
        return `You said: ${userText}. This voice is Rime's Coda model.`;
      }

      async function speak(text) {
        const res = await fetch("/api/tts", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({ text }),
        });
        if (!res.ok) throw new Error(`TTS failed: ${res.status}`);
        const player = document.getElementById("player");
        player.src = URL.createObjectURL(await res.blob());
        await player.play();
      }

      function log(line) {
        const li = document.createElement("li");
        li.textContent = line;
        document.getElementById("messages").appendChild(li);
      }

      document.getElementById("speak").addEventListener("click", () => {
        const SR = window.SpeechRecognition ?? window.webkitSpeechRecognition;
        if (!SR) return alert("SpeechRecognition not supported in this browser.");
        const rec = new SR();
        rec.onresult = async (e) => {
          const userText = e.results[0][0].transcript;
          const reply = respond(userText);
          log(`You: ${userText}`);
          log(`Agent: ${reply}`);
          await speak(reply);
        };
        rec.start();
      });
    </script>
  </body>
</html>

Open http://localhost:3000, select Speak, say something, and the agent answers in Rime’s astra voice.

Add streaming when latency matters

For incremental synthesis, attach a WebSocket bridge to the same HTTP server: browser WS ↔ Express server ↔ wss://users-ws.rime.ai/ws3 with header authentication. Audio can then start while later sentences are still generating. Browser WebSockets cannot send the required Authorization header, so the bridge stays server-side. The Next.js guide provides the complete bridge; it works with Express’s HTTP server through const server = app.listen(3000). The Coda WebSocket reference defines the /ws3 schema.

Production building blocks

WebSocket API overview

Endpoints, word-level timestamps, context IDs, and interruption handling.

Voices

Swap astra for any Coda voice. Coda covers eight languages, and each voice serves one of them.

Streaming formats

Choose between Opus, MP3, WAV, PCM, and μ-law for your latency budget.

Plain Node (no framework)

Build the same starter with only node:http and built-in fetch.

Overview

Quickstart

Voice agents

Voices & models

Customizing speech

Streaming & performance

Integrations

On-prem

Account & updates

Build a voice agent (Express)

Prerequisites

1. Server: Express app with a TTS route

2. Client: mic in, Rime audio out

Add streaming when latency matters

Production building blocks

WebSocket API overview

Voices

Streaming formats

Plain Node (no framework)

​Prerequisites

​1. Server: Express app with a TTS route

​2. Client: mic in, Rime audio out

​Add streaming when latency matters

​Production building blocks

WebSocket API overview

Voices

Streaming formats

Plain Node (no framework)

Prerequisites

1. Server: Express app with a TTS route

2. Client: mic in, Rime audio out

Add streaming when latency matters

Production building blocks