Metrics

Observability metrics exposed by Rime on-prem containers.

Rime services expose several endpoints for monitoring the health of a self-hosted deployment. Use them to inform scaling and performance decisions.

API container

The API container serves a /health route on port 8000 that reports overall status:

curl -X GET "http://localhost:8000/health"

A typical response looks like this:

{
  "apiStatus": "ok",
  "timestamp": "2025-04-03T13:59:58.902Z",
  "licenseStatus": "valid",
  "modelReachable": true
}

The response confirms that both the API and model services are up and responding.

Arcana container

Arcana’s cloud service is retiring on August 15, 2026. On-prem Arcana images stay available to pull and run, but stop receiving updates and support after that date, including vulnerability patches. See Migrate from Arcana to Coda for what changes if you move.

Healthcheck

To check if the model is running properly, you can perform a liveness probe using the /readyz endpoint:

curl -X GET "http://localhost:8080/readyz"

A typical response looks like this:

ok

Metrics

The model can emit OpenTelemetry metrics.

The internal vLLM process also exposes a Prometheus /metrics endpoint.

You can use either as a data source for metrics collection, tuning, and debugging.

OpenTelemetry metrics

The Rime engine exposes the following OpenTelemetry metrics:

• rime.engine.concurrent_pipeline
• rime.engine.generated_audio_duration
• rime.engine.gpu_load
• rime.engine.initial_latency
• rime.engine.invocation_request

To have the model emit OpenTelemetry metrics, set two environment variables …

• … set OTEL_COLLECTOR_PROTOCOL to grpc (recommended), http/protobuf, or http/json.
• … provide a valid OpenTelemetry Collector endpoint (through OTEL_COLLECTOR_ENDPOINT).

e.g.;

OTEL_COLLECTOR_PROTOCOL='grpc'
OTEL_COLLECTOR_ENDPOINT="https://${OTELCOL_DOMAIN:?}:4317"

The model does not set any resource attributes for these metrics. OTEL_RESOURCE_ATTRIBUTES may be used to define them.

The model does not implement OpenTelemetry authentication. If you need authentication, run an OpenTelemetry Collector sidecar to forward the metrics.

Prometheus metrics

To retrieve vLLM Prometheus metrics, issue an HTTP request to http://localhost:${GENERATOR_VLLM_PORT:?}/metrics.

By default, a random port is assigned to the vLLM server. To choose a fixed port (which would be convenient for forwarding these metrics, if desired), set the environment variable GENERATOR_VLLM_PORT to an available port number.

OpenTelemetry Collector sidecar

If you need authentication or other OpenTelemetry Collector extensions, use the contrib distribution that includes them.

You can run an OpenTelemetry Collector to forward both the OpenTelemetry and Prometheus metrics, or to retrieve the OpenTelemetry metrics locally.

For example, Docker Compose can be used to run the model and an OpenTelemetry Collector …

services:
  model:
    image: us-docker.pkg.dev/rime-labs/arcana/v2:20251206
    # ...
    environment:
      # ...
      "GENERATOR_VLLM_PORT": 30000
      "OTEL_COLLECTOR_PROTOCOL": "grpc"
      "OTEL_COLLECTOR_ENDPOINT": "http://otelcol:4317"
  otelcol:
    image: otel/opentelemetry-collector-contrib:0.139.0
    ports:
      # ...
      - "9090:9090" # Prometheus exporter, for local retrieval on a /metrics endpoint
    volumes:
      - "./otelcol/config.yaml:/etc/otelcol-contrib/config.yaml:ro"
      # ...

… where otelcol/config.yaml would be the configuration file for the OpenTelemetry Collector.

receivers:
  # Provide endpoints for OpenTelemetry metrics from the model
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  # Scrape the vLLM Prometheus /metrics endpoint
  prometheus/generator-vllm:
    config:
      global:
        scrape_interval: "1s"
      scrape_configs:
        - job_name: "generator-vllm"
          static_configs:
            # Assuming GENERATOR_VLLM_PORT=30000
            - targets: ["model:30000"]
exporters:
  otlp:
    # Consider using the authenticator extension if authentication is required
    endpoint: "another.opentelemetry.collector.example:4317"
  prometheus:
    endpoint: "0.0.0.0:9090"
service:
  pipelines:
    # Forward the OpenTelemetry and Prometheus metrics to another OpenTelemetry Collector endpoint
    metrics/otlp:
      receivers: [otlp, prometheus/generator-vllm]
      exporters: [otlp]
    # Expose the OpenTelemetry and Prometheus metrics locally with another Prometheus /metrics endpoint
    metrics/local:
      receivers: [otlp, prometheus/generator-vllm]
      exporters: [prometheus]

Histogram tuning

Metrics that emit histogram data use pre-configured bucket boundaries. The Rime engine prescribes its own defaults for some metrics. These may also be configurable/tunable through the use of HISTOGRAM_BUCKETS_* environment variables.

To separate differently-tuned sets of data across individual metrics, a suffix may be attached to the original metric name using HISTOGRAM_SUFFIX_* environment variables.

• rime.engine.initial_latency

  • Measured in milliseconds.
  • Default bounds:
    • 0.0, 100.0, 150.0, 200.0, 225.0, 250.0, 275.0, 300.0, 325.0, 350.0, 375.0, 400.0, 450.0, 500.0, 1000.0
      
  • Set alternative bounds:
    • HISTOGRAM_BUCKETS_INITIAL_LATENCY_MS='0.0, 90.0, 95.0, 100.0, 105.0, 110.0, 115.0, 120.0, 125.0, 130.0, 135.0, 140.0, 150.0, 175.0, 200.0'
      
  • Set alternative metric name suffix:
    • HISTOGRAM_SUFFIX_INITIAL_LATENCY='tuned_120ms' produces a metric named rime.engine.initial_latency.tuned_120ms

• rime.engine.generated_audio_duration

  • Measured in seconds.
  • Default bounds:
    • 0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, 6.0, 7.5, 10.0, 20.0, 30.0, 60.0
      
  • Set alternative bounds:
    • HISTOGRAM_BUCKETS_GENERATED_AUDIO_DURATION_SECONDS='0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, 6.0, 7.5, 10.0, 20.0, 30.0, 60.0'
      
  • Set alternative metric name suffix:
    • HISTOGRAM_SUFFIX_GENERATED_AUDIO_DURATION='tuned_3s' produces a metric named rime.engine.generated_audio_duration.tuned_3s

Mist container

Healthcheck

To check if the model is running properly, use the /livez and /readyz probes on the engine HTTP port:

curl -X GET "http://localhost:8080/livez"
curl -X GET "http://localhost:8080/readyz"

Current Mist images do not serve /ping. Use /livez for liveness and /readyz for readiness.

Metrics

Prometheus-compatible metrics are served by the generator, not on the engine HTTP port. Pin the port with GENERATOR_SERVICE_PORT on the model container and scrape it there:

curl -X GET "http://localhost:30000/metrics"

See Prometheus for a complete scrape configuration.

This endpoint provides telemetry data, including:

• HTTP request counters: requests broken down by endpoint, status code, and HTTP method
• Error tracking: counts of HTTP errors by type and status code

Example metrics include:

# HELP http_requests_total Total number of HTTP requests
# TYPE http_requests_total counter
http_requests_total{endpoint="/invocations",http_status="200",method="POST"} 102018.0
http_requests_total{endpoint="/metrics",http_status="200",method="GET"} 69908.0
http_requests_total{endpoint="/invocations",http_status="500",method="POST"} 38.0

# HELP http_errors_total Total number of HTTP errors
# TYPE http_errors_total counter
http_errors_total{endpoint="/invocations",error_message="cannot access local variable '_var_var_6' where it is not associated with a value",http_status="500",method="POST"} 38.0

Scrape these metrics with Prometheus to build dashboards and alerts for your deployment.