View By Selector
| View by | Groups metrics by | When to use |
|---|---|---|
| Configs | Configuration / rule name (default) | See which routing configs, rate limits, and budget limits are being triggered |
| Users | Username of the caller | Identify users hitting rate or budget limits |
| Virtual Accounts | Virtual account | Monitor policy impact by application |
| Teams | Team name | Understand policy impact per team |
Top-Level Counters
Three headline metrics are displayed at the top:- Total Loadbalances — number of times load balancing was applied to distribute requests across models or deployments.
- Model Calls Blocked By Rate Limit — total requests rejected because they exceeded a rate limit.
- Model Calls Blocked By Budget Limit — total requests rejected because they exceeded a budget constraint.
Routing Charts
Routing Rule Usage Rate
Shows how often each routing rule is triggered over time. This helps you understand which rules are actively routing traffic and whether the distribution matches your expectations.Routing Failure Rate
Routing-level failures over time. A routing failure means the gateway could not find a valid target for a request — this could happen if all backends in a routing rule are unavailable or if a fallback chain is exhausted.Routing Rule Target Model Breakdown
A horizontal bar chart showing which target models each routing rule resolves to. This is invaluable for verifying that:- Load balancing is distributing traffic as configured.
- Fallback rules are directing traffic to the intended backup models.
- Routing changes are taking effect as expected.
Rate Limiting Charts
Rate Limit Checks Rate
Shows the rate of rate-limit evaluations over time. Each incoming request that matches a rate limit rule triggers a check. This tells you how much traffic is subject to rate limiting.Rate Limit Exceeded Rate
How often rate limits are actually being hit. If this is consistently high for a specific rule, it may mean:- The limit is too restrictive and needs to be increased.
- A specific user or application is consuming more than its fair share.
- You need to distribute load more evenly across users or virtual accounts.
Rate Limit Result Breakdown
A horizontal bar chart showing allowed vs. blocked counts per rate-limit rule. This gives you a clear picture of how much traffic is being throttled by each rule.Budget Limit Charts
Budget Limit Checks Rate
Shows the rate of budget-limit evaluations over time. Budget limits control how much spending is allowed per virtual account, user, or team.Budget Limit Exceeded Rate
How often budget limits are being hit. A budget limit that is frequently exceeded may need adjustment, or it may indicate a user or application that needs to be onboarded to a higher tier.Budget Limit Result Breakdown
A horizontal bar chart showing allowed vs. blocked counts per budget rule. This helps you understand:- Which budget rules are the most restrictive.
- Whether blocked requests are concentrated on specific accounts.
- Whether budget allocations need rebalancing.
Common Use Cases
- Verify a routing change: After updating a routing rule, check the Routing Rule Target Model Breakdown to confirm traffic is flowing to the intended models.
- Tune rate limits: If users are complaining about being throttled, check the Rate Limit Exceeded Rate and Result Breakdown to see if limits need adjustment.
- Monitor budget consumption: Use the Budget Limit charts to proactively identify accounts approaching their limits before they start getting blocked.
- Diagnose fallback behavior: If a primary model is down and fallbacks are kicking in, the Routing Rule Usage Rate and Target Model Breakdown will show the shift in traffic.
- Capacity planning: Use the Routing Rule Usage Rate to understand how load is distributed and plan capacity for high-traffic routing rules.