Build a voice agent (Vite)

Build a minimal voice-agent loop in Vite while a small Node proxy keeps your Rime API key off the browser.

Build a minimal voice-agent loop in Vite without exposing your API key. Because Vite serves static assets, a small Node server proxies synthesis while the browser handles speech input and audio playback.

Call Rime over HTTPS with fetch; no Rime-specific npm package is required. This starter uses browser speech recognition and a stub response function, both of which you should replace for production.

Browser (mic + playback)  →  /api/tts (Vite dev proxy)  →  Node proxy :3001  →  https://users.rime.ai/v1/rime-tts

Prerequisites

• A Rime API key from the API Tokens page, exported as RIME_API_KEY
• A Vite + React project (npm create vite@latest -- --template react-ts defaults work)
• Express for the proxy: npm install express

1. The TTS proxy server

Create server.mjs in the project root:

server.mjs

import express from "express";

const app = express();
app.use(express.json());

app.post("/api/tts", async (req, res) => {
  const { text, speaker = "astra", modelId = "coda" } = req.body ?? {};
  if (!text) return res.status(400).json({ error: "text is required" });

  const rimeRes = await fetch("https://users.rime.ai/v1/rime-tts", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.RIME_API_KEY}`,
      "Content-Type": "application/json",
      Accept: "audio/mpeg", // MP3; see the API reference for Opus, WAV, PCM, μ-law
    },
    body: JSON.stringify({ text, speaker, modelId }),
  });

  if (!rimeRes.ok) {
    return res
      .status(rimeRes.status)
      .json({ error: `Rime API error ${rimeRes.status}`, detail: await rimeRes.text() });
  }

  res.setHeader("Content-Type", "audio/mpeg");
  res.send(Buffer.from(await rimeRes.arrayBuffer()));
});

app.listen(3001, () => console.log("Rime TTS proxy on http://localhost:3001"));

Wire the Vite dev server to it in vite.config.ts, so the browser can call a same-origin /api/tts:

vite.config.ts

import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";

export default defineConfig({
  plugins: [react()],
  server: {
    proxy: {
      "/api": "http://localhost:3001",
    },
  },
});

Run both and verify the proxy before touching the UI:

RIME_API_KEY=your_key node server.mjs &
npm run dev
curl -X POST http://localhost:5173/api/tts \
  -H 'Content-Type: application/json' \
  -d '{"text": "Hello from my Vite voice agent."}' \
  --output test.mp3

test.mp3 should be playable audio. A 401 means RIME_API_KEY isn’t visible to server.mjs.

2. Client: mic in, Rime audio out

Replace src/App.tsx. It uses the browser’s built-in SpeechRecognition for input (Chrome/Edge/Safari), a stub respond() function as the agent brain, and the proxy for the voice:

src/App.tsx

import { useRef, useState } from "react";

// Replace this stub with a call to your LLM of choice.
function respond(userText: string): string {
  return `You said: ${userText}. This voice is Rime's Coda model.`;
}

export default function App() {
  const [messages, setMessages] = useState<string[]>([]);
  const [listening, setListening] = useState(false);
  const audioRef = useRef<HTMLAudioElement>(null);

  async function speak(text: string) {
    const res = await fetch("/api/tts", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ text }),
    });
    if (!res.ok) throw new Error(`TTS failed: ${res.status}`);
    const url = URL.createObjectURL(await res.blob());
    audioRef.current!.src = url;
    await audioRef.current!.play();
  }

  async function handleUserText(userText: string) {
    const reply = respond(userText);
    setMessages((m) => [...m, `You: ${userText}`, `Agent: ${reply}`]);
    await speak(reply);
  }

  function listen() {
    const SR =
      (window as any).SpeechRecognition ??
      (window as any).webkitSpeechRecognition;
    if (!SR) return alert("SpeechRecognition not supported in this browser.");
    const rec = new SR();
    rec.onresult = (e: any) => handleUserText(e.results[0][0].transcript);
    rec.onend = () => setListening(false);
    setListening(true);
    rec.start();
  }

  return (
    <main style={{ maxWidth: 480, margin: "4rem auto", fontFamily: "sans-serif" }}>
      <h1>Rime voice agent</h1>
      <button onClick={listen} disabled={listening}>
        {listening ? "Listening…" : "🎤 Speak"}
      </button>
      <ul>{messages.map((m, i) => <li key={i}>{m}</li>)}</ul>
      <audio ref={audioRef} hidden />
    </main>
  );
}

Open http://localhost:5173, select Speak, say something, and the agent answers in Rime’s astra voice.

In production, deploy server.mjs or fold the route into your existing backend, then serve Vite’s built assets from it. The same-origin /api/tts path keeps working, and the API key remains in the server environment.

Add streaming when latency matters

For incremental synthesis, connect your server to Rime’s WebSocket API and relay events to the browser. Audio can then start while later sentences are still generating. Browser WebSockets cannot send the required Authorization header, so the bridge stays server-side. The Next.js guide shows the bridge pattern; the Coda WebSocket reference defines the /ws3 message schema.

Production building blocks

WebSocket API overview

Endpoints, word-level timestamps, context IDs, and interruption handling.

Voices

Swap astra for any Coda voice. Coda covers eight languages, and each voice serves one of them.

Streaming formats

Choose between Opus, MP3, WAV, PCM, and μ-law for your latency budget.

TTS in five minutes

The core API walkthrough in cURL, Python, JavaScript, and TypeScript.