Blank white background with no objects or features visible.

Meet TrueForge: The open-source, vendor-neutral agent harness. 50% lower cost. Explore Now→

97.8M
AI requests routed through the Gateway in July 2026
99.995%
model request success rate in production

Staffbase is the first AI-native Employee Experience Platform. It brings communications, IT, and HR together to reach every employee — including the front-line workers who are often the hardest to connect with — through a branded employee app, intranet, email, SMS, digital signage, Microsoft 365 integrations, and new agentic experiences such as personalized podcasts and a conversational assistant. More than 1,500 enterprise customers, including Adidas, Alaska Airlines, DHL, MAN Truck & Bus, and Whataburger, use Staffbase to reach their people. Named a Leader in the 2025 Gartner® Magic Quadrant™ for Intranet Packaged Solutions for the third consecutive year and recognized by G2 as a Leader for Employee Intranets, the company is headquartered in New York City and Chemnitz, Germany.

Being AI-native means AI isn’t a bolt-on feature at Staffbase — it is the foundation the product is built on. Every new experience, from semantic search across the intranet to the conversational assistant to personalized podcasts, depends on running language-model and embedding inference reliably, globally, and at enterprise scale. This case study looks at how Staffbase used the TrueFoundry AI Gateway to unify that inference across a sprawling multi-region cloud footprint and turn it into a single, dependable platform.

The challenge: enterprise-scale AI across a sprawling cloud footprint

As Staffbase pushed AI into the core of its platform, the hard problem wasn’t calling a model — it was running inference reliably at the scale and across the geography an enterprise employee-experience platform demands.

Three pressures stood out. First, raw scale: an AI-native platform serving 1,500+ enterprise customers generates enormous, continuous inference volume, dominated by the embedding calls that power search and retrieval across every customer’s content. Second, a fragmented provider and region footprint: capacity was spread across multiple Azure subscriptions and cognitive accounts — separate deployments in regions like Sweden and France, each with its own API keys, quotas, and rate limits — plus AWS Bedrock. Balancing traffic across all of them by hand, and staying inside each account’s rate limits, would have been untenable. Third, data residency and reliability: with a European headquarters and global customers, Staffbase needed inference routed through the right regions, and — because employee communications are mission-critical — it needed uptime it could prove.

"Managing AI resources across multiple regions and subscriptions had become too complex to handle manually. We didn't want our engineers thinking about which cognitive account had quota left, or which region a request should land in — we needed one endpoint that just handled it. We were able to manage that with TrueFoundry."

Stefan Staude, Senior Site Reliability Engineer

The solution: one gateway across every subscription, region, and model

Staffbase standardized on the TrueFoundry AI Gateway as the single control plane for all of its AI traffic. Behind one endpoint, the Gateway now fronts a deliberately complex backend — and hides that complexity from every team building on it. 

1. A unified proxy across 141 deployments and 19 models

The Gateway gives Staffbase one interface to a backend that spans 5 Azure subscriptions plus AWS Bedrock, 28 Azure cognitive accounts, and 141 individual model deployments across 9 regions. Teams call a single API and get access to 19 models — from the latest frontier models to specialized embedding and reranking models — without wiring up provider SDKs, keys, or endpoints themselves. 

Gateway Azure subs Cognitive accounts Regions Routing targets Models
de1 / prod 1 6 4 32 18
us1 / prod 1 5 3 27 18
au1 / prod 1 6 5 23 17
de1 / stage 1 6 4 31 18
de1 / dev 1 + AWS 5 + Bedrock 3 + 1 28 19
Total 5 Azure + 1 AWS 28 Azure accounts 9 regions 141

Available models (19): gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra, gpt-5.5, gpt-5.4, gpt-5.1, gpt-5-mini, gpt-5-nano, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4o, claude-sonnet-5, claude-sonnet-4-6, claude-haiku-4-5, Mistral-Large-3, text-embedding-3-large, text-embedding-3-small, cohere-rerank-v3-5.

2. Intelligent load balancing across subscriptions and regions

This is where the Gateway earns its keep. Staffbase’s Azure capacity is split across separate cognitive accounts — Sweden and France, for example, sit in the same subscription but carry their own API keys, quotas, and rate limits. The Gateway pools all 141 deployment targets into one logical capacity and routes each request to a deployment that has headroom, automatically working around per-account rate limits and failing over when one endpoint is saturated. The de1 production gateway alone balances across 32 routing targets; across all environments the Gateway manages 141. What used to be a manual capacity-planning problem is now something engineers never have to think about.

“The routing and load balancing across our Azure accounts is the single biggest thing the Gateway does for us. Sweden, France, every deployment gets pooled into one endpoint, and traffic just flows to wherever there’s quota. We prevented hitting rate limits and stopped babysitting capacity.”

Stefan Staude, Senior Site Reliability Engineer

3. Multi-region routing for data residency and latency

Production runs as distinct regional gateways — prod-de1 (Europe), prod-us1 (US), and prod-au1 (Australia) — so requests are served from the right geography for both data-residency and latency. For a platform with a European headquarters and enterprise customers across the globe, routing inference through region-specific gateways keeps data where it belongs while giving each region its own dependable capacity pool.

4. Embeddings at massive scale

The traffic mix tells the story of an AI-native product: embeddings account for over 97% of all Gateway traffic, powering the retrieval and semantic search behind Staffbase’s AI experiences. Since go-live in February, the Gateway has processed roughly 108.2M requests to text-embedding-3-small and 29.3M to text-embedding-3-large, alongside 2.5M generation requests to gpt-5-mini and growing volume across the frontier models.

Model (since Feb, ~177 days) Total requests
text-embedding-3-small ~108.2M
text-embedding-3-large ~29.3M
gpt-5-mini ~2.5M

5. Reliability you can measure

Because employee communications can’t afford downtime, Staffbase tracks the Gateway against formal SLOs. Over a 28-day rolling window, availability sits at up to 99.99998% — comfortably past five nines — across every production environment, and model-request success rates hold at 99.99%+ and above once expected setup errors (rate-limit and auth responses) are excluded.

Environment Availability (28d) Success rate (30d)
prod-de1 99.99998% 99.995%
prod-us1 99.99987% 99.995%
prod-au1 99.99674% 99.990%
dev 99.99981% 99.990%
stage-de1 99.99976% 99.984%

6. Per-feature and per-team cost visibility

Around 22 distinct product features and teams route their traffic through the Gateway today, and every one of the 12 features Staffbase shipped between February and May 2026 depends on it. Because all of that traffic flows through one governed surface, Staffbase gets a single, attributable view of usage and spend — for a fraction of what the volume suggests. 

Support that keeps pace with a fast-moving roadmap

Adopting a platform this deeply only works if the vendor moves at the team’s speed. Staffbase highlighted TrueFoundry’s responsiveness — particularly how quickly feature requests are turned around — as a real advantage while it scales its AI adoption.

Results: 140M+ governed requests at 99.99%+ reliability

By centralizing on TrueFoundry, Staffbase turned a fragmented, multi-subscription, multi-region AI footprint into a single governed platform its teams can build on without friction.

Since going live in February, the Gateway has governed more than 141 million requests — now around 97.8M in a typical month — and processed 29.2 billion tokens (27.3B input and 1.9B output). It has held availability at up to 99.99998% and success rates at 99.99%+ while balancing across 141 deployments in 9 regions, all behind one endpoint. Roughly 22 features and teams rely on it, and every AI feature Staffbase shipped in 2026 runs on top of it.

“The Gateway became the layer every AI feature is built on. It gave us one place to run inference at scale, across every region and provider, with the reliability our customers expect — and it did it without our teams having to become infrastructure experts.”

Luca Ghersi, Director of Engineering @ Staffbase

Key takeaways for teams adopting an AI gateway

  • Pool your capacity, don’t manage it by hand. When inference is spread across many subscriptions, accounts, and regions, a gateway that pools them into one logical capacity and routes around rate limits removes an entire class of operational toil.
  • Design for the workload you actually have. For an AI-native product, embeddings can dominate traffic — over 97% here. A gateway that routes high-volume, low-cost calls efficiently keeps spend far below what raw request counts imply.
  • Make reliability measurable. Tracking the gateway against formal SLOs turned uptime from a hope into a number Staffbase can stand behind for mission-critical communications.
  • Centralize first, then attribute. One governed endpoint for ~22 features gave Staffbase a single view of usage and cost, attributable by feature and team, instead of reconciling spend across providers.

Conclusion

Staffbase’s experience shows that being AI-native at enterprise scale doesn’t require building inference infrastructure from scratch. By standardizing on TrueFoundry’s AI Gateway for unified inference, intelligent multi-region load balancing, and cost visibility — across 5 Azure subscriptions, AWS Bedrock, 28 cognitive accounts, and 141 deployments — Staffbase scaled to more than 140 million+ governed requests at five-nines reliability, while giving every team one dependable place to build the next generation of employee experiences.

The fastest way to build, govern and scale your AI

Operate your ML Pipeline from Day 0

pipeline