Applicable to: Users who have dashboards, alerts, or any monitoring pipelines that consume AI Gateway metrics with the llm_gateway_* prefix.
What Is Changing
All metric names in AI Gateway are being renamed from the llm_gateway_* prefix to ai_gateway_*.
The general rule is a prefix swap: llm_gateway_ → ai_gateway_. Two metrics differ from this rule: llm_gateway_request_total_cost_total → ai_gateway_request_cost_total, and llm_gateway_agent_request_tool_calls_total → ai_gateway_agent_request_tool_calls.
Complete rename map (names as seen in Prometheus scrape):
Core request metrics
| Old Name | New Name |
|---|
llm_gateway_requests_total | ai_gateway_requests_total |
llm_gateway_input_tokens_total | ai_gateway_input_tokens_total |
llm_gateway_output_tokens_total | ai_gateway_output_tokens_total |
llm_gateway_request_total_cost_total | ai_gateway_request_cost_total |
llm_gateway_request_model_inference_failure_total | ai_gateway_request_model_inference_failure_total |
llm_gateway_config_parsing_failures_total | ai_gateway_config_parsing_failures_total |
llm_gateway_request_processing_ms | ai_gateway_request_processing_ms |
llm_gateway_first_token_latency_ms | ai_gateway_first_token_latency_ms |
llm_gateway_inter_token_latency_ms | ai_gateway_inter_token_latency_ms |
| Old Name | New Name |
|---|
llm_gateway_agent_requests_total | ai_gateway_agent_requests_total |
llm_gateway_agent_request_duration_ms | ai_gateway_agent_request_duration_ms |
llm_gateway_agent_request_tool_calls_total | ai_gateway_agent_request_tool_calls |
llm_gateway_agent_request_iteration_limit_reached_total | ai_gateway_agent_request_iteration_limit_reached_total |
llm_gateway_agent_llm_latency_ms | ai_gateway_agent_llm_latency_ms |
llm_gateway_agent_tool_calls_total | ai_gateway_agent_tool_calls_total |
llm_gateway_agent_tool_latency_ms | ai_gateway_agent_tool_latency_ms |
llm_gateway_agent_mcp_connect_latency_ms | ai_gateway_agent_mcp_connect_latency_ms |
| Old Name | New Name |
|---|
llm_gateway_load_balanced_requests_total | ai_gateway_load_balanced_requests_total |
llm_gateway_fallback_requests_total | ai_gateway_fallback_requests_total |
llm_gateway_rate_limit_applied_total | ai_gateway_rate_limit_applied_total |
llm_gateway_rate_limit_nominated_total | ai_gateway_rate_limit_nominated_total |
llm_gateway_rate_limit_requests_total (deprecated) | ai_gateway_rate_limit_requests_total (deprecated) |
| Old Name | New Name |
|---|
llm_gateway_budget_applied_total | ai_gateway_budget_applied_total |
llm_gateway_budget_nominated_total | ai_gateway_budget_nominated_total |
llm_gateway_budget_usage | ai_gateway_budget_usage |
llm_gateway_budget_requests_total (deprecated) | ai_gateway_budget_requests_total (deprecated) |
| Old Name | New Name |
|---|
llm_gateway_guardrails_requests_total | ai_gateway_guardrails_requests_total |
llm_gateway_guardrail_requests_total | ai_gateway_guardrail_requests_total |
llm_gateway_guardrail_execution_latency_ms | ai_gateway_guardrail_execution_latency_ms |
| Old Name | New Name |
|---|
llm_gateway_model_cooldown_state_total | ai_gateway_model_cooldown_state_total |
llm_gateway_model_threshold_latency_exceeded_total | ai_gateway_model_threshold_latency_exceeded_total |
Why This Change
The product has been rebranded from LLM Gateway to AI Gateway to reflect the broader set of capabilities it now supports (agents, MCP, guardrails, etc.). Aligning the OTel metric names with the product name makes monitoring consistent with the rest of the platform.
Transition: Dual-Write Mode
To avoid breaking existing dashboards and alerts immediately, a dual-write mode was introduced in v0.136.0. When enabled, the /metrics endpoint emits both the old llm_gateway_* metrics and the new ai_gateway_* metrics in a single scrape response.
Dual-write is controlled by the environment variable:
ENABLE_DOUBLE_WRITE_FOR_METRICS=true # default: true (since v0.136.0)
The old llm_gateway_* metrics will no longer be published after 30th June 2026. Ensure your dashboards and alerts are migrated to the new ai_gateway_* names before this date.
What You Need to Do
Verify dual-write is active
Confirm that ENABLE_DOUBLE_WRITE_FOR_METRICS is set to true (it is true by default). Scrape the /metrics endpoint and verify you see both llm_gateway_* and ai_gateway_* series.
Update dashboards and alerts
Replace all references to llm_gateway_* OTel metric names with the corresponding ai_gateway_* names in your Grafana dashboards, Prometheus alerting rules, and any other monitoring configuration.Use the rename table above as your migration reference. The general pattern is:
- Prefix:
llm_gateway_ → ai_gateway_ (only change visible in the scraped output — counters retain _total as the Prometheus exporter re-adds it)
Disable dual-write after migration
Once all dashboards and alerts have been updated to use the new ai_gateway_* names, set ENABLE_DOUBLE_WRITE_FOR_METRICS=false to stop emitting the legacy metrics and reduce scrape payload size.
If you have questions or need help with the migration, reach out to support@truefoundry.com — we’re happy to assist.
