Metrics

Health and OpenTelemetry metrics exposed by supported Rime on-prem containers.

Check the engine directly in a standalone licensed deployment. This setup has no API container to serve /health.

HTTP health

Use /livez to check that the health service responds. Use /readyz to check that the engine can accept inference requests. In the model image, the engine serves both endpoints on port 8080:

curl --fail http://localhost:8080/livez
curl --fail http://localhost:8080/readyz

Readiness returns HTTP 200 with ok when the engine is ready. It returns HTTP 503 during loading, quiet mode (after the engine receives SIGUSR2), or drain. Liveness can succeed while readiness fails. Use readiness to route traffic and liveness to decide on restarts, so a slow model load doesn’t trigger a restart.

gRPC health

On port 7463, the engine serves the standard grpc.health.v1.Health/Check and Watch methods. Check overall readiness with the empty service name, "". Send traffic when the result is SERVING and withdraw it when the result is NOT_SERVING.

Install grpcurl and place the standard health schema at ./rime-protos/grpc/health/v1/health.proto. This command checks readiness on the private local listener:

grpcurl -plaintext \
  -import-path ./rime-protos -proto grpc/health/v1/health.proto \
  -d '{"service":""}' localhost:7463 grpc.health.v1.Health/Check

Keep the schema argument. The engine does not expose reflection, so grpcurl needs the schema to make the request. See Engine protocols for synthesis schemas and client setup.

OpenTelemetry metrics

The Rime engine exposes these metrics:

• rime.engine.active_inference_request
• rime.engine.inference_concurrency_capacity
• rime.engine.application_utilization
• rime.engine.inference_admission_rejected
• rime.engine.concurrent_pipeline
• rime.engine.generated_audio_duration
• rime.engine.gpu_load
• rime.engine.initial_latency
• rime.engine.invocation_request
• rime.engine.unicode_scalar_value_count
• rime.engine.stage.items_in
• rime.engine.stage.items_out
• rime.engine.stage.first_output_latency

The rime.engine.stage.* series carry a stage attribute.

Set OTEL_COLLECTOR_PROTOCOL to grpc (recommended), http/protobuf, or http/json, and set OTEL_COLLECTOR_ENDPOINT to your collector endpoint:

OTEL_COLLECTOR_PROTOCOL=grpc
OTEL_COLLECTOR_ENDPOINT=http://otelcol:4317

The model does not set resource attributes. Use OTEL_RESOURCE_ATTRIBUTES to add them. The model also does not implement OpenTelemetry authentication, so run a collector sidecar when the upstream endpoint requires credentials.

To expose the OpenTelemetry series for Prometheus, follow the Prometheus integration guide.

Logs

The model container writes one plain-text line per event to its standard output. Request text is not included by default: lines that describe a request carry text_chars, the length of the input, instead of the input itself.

Set RIME_LOG_REQUEST_TEXT=true to include request text in those lines. Do this only where the container log stays inside a boundary that may hold end-user text, because anything reading the log, including a log forwarder, receives the text verbatim.

RIME_LOG_REQUEST_TEXT=true

Set OTEL_LOGS_EXPORTER=otlp to have the model also send every log line to the collector named by OTEL_COLLECTOR_ENDPOINT, over OTEL_COLLECTOR_PROTOCOL. Both variables must be set as for metrics, or the switch exports nothing. The model sends each line as one OTLP log record: the line is the body, the log level is the severity, and request_id is an attribute on lines that belong to a request. The model sends logs nowhere else, and never to Rime.

OTEL_LOGS_EXPORTER=otlp

Leave OTEL_LOGS_EXPORTER unset to keep logs on standard output only. The collector endpoint then carries metrics alone.

Histogram tuning

Metrics that emit histogram data use preconfigured bucket boundaries. Override them with the corresponding HISTOGRAM_BUCKETS_* environment variable. Add a separate suffix with HISTOGRAM_SUFFIX_* when you need to distinguish differently tuned series.

Initial latency

rime.engine.initial_latency is measured in milliseconds.

Default bounds:

0.0, 100.0, 150.0, 200.0, 225.0, 250.0, 275.0, 300.0, 325.0, 350.0, 375.0, 400.0, 450.0, 500.0, 1000.0

Example overrides:

HISTOGRAM_BUCKETS_INITIAL_LATENCY_MS='0.0, 90.0, 95.0, 100.0, 105.0, 110.0, 115.0, 120.0, 125.0, 130.0, 135.0, 140.0, 150.0, 175.0, 200.0'
HISTOGRAM_SUFFIX_INITIAL_LATENCY='tuned_120ms'

The suffix produces rime.engine.initial_latency.tuned_120ms.

Generated audio duration

rime.engine.generated_audio_duration is measured in seconds.

Default bounds:

0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, 6.0, 7.5, 10.0, 20.0, 30.0, 60.0

Example overrides:

HISTOGRAM_BUCKETS_GENERATED_AUDIO_DURATION_SECONDS='0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, 6.0, 7.5, 10.0, 20.0, 30.0, 60.0'
HISTOGRAM_SUFFIX_GENERATED_AUDIO_DURATION='tuned_3s'

The suffix produces rime.engine.generated_audio_duration.tuned_3s.