What is OpenRouter? A technical guide to what happens to your request

By Kshitij Gupta · Published October 7, 2026 · Updated October 15, 2026

TL;DR: OpenRouter is a hosted, OpenAI-compatible API that sends a team's model requests to hundreds of third-party providers through one endpoint and one key, and bills the team at provider list prices plus a fee on credit purchases. The model name a team sends doesn't decide who serves it; routing does. TrueFoundry followed a request through that path and measured what it adds.

Teams that need prompts to stay on their own network, machine identities instead of person-owned keys, or their own provider contracts move to a gateway they run, such as the TrueFoundry AI Gateway, which runs in the team's VPC, on-prem or air-gapped. Moving does not mean giving up OpenRouter's catalog. OpenRouter is supported as a provider in TrueFoundry, so the catalog can sit behind the team's own rate limits, budgets, access control and logging. For teams already on OpenRouter, that is usually the lowest-risk first step.

Five model providers means five SDKs, five API keys, five invoices and five sets of rate limits. OpenRouter collapses that into one endpoint. Point the OpenAI SDK at a different base URL, change the model string, and the same client can call Claude, Gemini, Llama or DeepSeek.

That much is in every quickstart. Underneath, OpenRouter is a marketplace. It picks which company serves each request, holds a team's money, and pays the providers. TrueFoundry traced what happens between a team's code and the model, with numbers from its own testing where it has them.

What is OpenRouter?

OpenRouter is a hosted HTTP API that routes inference requests to third-party model providers through a single endpoint and credential, and bills teams for model usage by default. It accepts three request formats: OpenAI Chat Completions, the OpenAI Responses API, and Anthropic's Messages API at /api/v1/messages. There are client SDKs for TypeScript, Python and Go, but they're a typed layer over the same REST API, so the plain OpenAI SDK with a changed baseURL works just as well.

OpenRouter is two-sided. On one side it recruits providers, sets the contract they integrate against, and pays them. On the other it holds the team's prepaid credits and consolidates billing across all of them. It sits in the team's payment path as well as the team's request path, which is what makes it convenient, and also why leaving takes more than changing a URL.

Teams are renting access to other companies' models. There is no self-hosted or air-gapped build of OpenRouter, and the core product is closed source. The SDKs and docs are public. It started as Window, a Chrome extension for using several models at once, and since then has added server-side tools such as web search, web fetch and sandboxed shell execution that models can call mid-request.

How a request flows through OpenRouter

One request goes through these steps:

  1. A team's app sends it to https://openrouter.ai/api/v1 with one API key, in any of the three formats.
  2. OpenRouter's edge, running on Cloudflare Workers, authenticates the key and checks the team's credit balance.
  3. The model string, in vendor/model form, resolves to a list of provider endpoints that serve that model.
  4. OpenRouter picks one. By default it skips providers with an outage in the last 30 seconds and weights the rest by the inverse square of their price. The team's sort, order, only and data-policy settings override that.
  5. The request is translated for that provider and forwarded, and the response is normalised back to the format the team sent.
  6. Usage is billed: deducted from the team's credits at the provider's list price, or charged to the team's own provider key if the team has attached one.
  7. If the provider fails, OpenRouter tries the next one. The attempt field in the response is the only in-band sign that happened.

One model name, eleven different deployments

The model string does not tell teams who runs the model. The vendor prefix says whose model it is, not whose servers it's on. When TrueFoundry tested anthropic/claude-3-haiku, it was served by Amazon Bedrock.

For open-weight models the gap gets wider. On 5 October, OpenRouter's API listed 11 endpoints for meta-llama/llama-3.3-70b-instruct. Across those 11:

Same name, materially different products. If the team's prompts run long, the 12K endpoint and the 131K endpoint aren't interchangeable, and the default router chooses partly on price. OpenRouter does filter by capability when a request needs it: when TrueFoundry sent eight requests with tool_choice: required and no provider pin, every one landed on a host that declares tool support. But declared isn't the same as working. One host that declares tool support returned no tool call at all when TrueFoundry pinned to it and forced one.

Read /api/v1/models/{model}/endpoints before trusting a model string, and pin or filter providers wherever quantisation, context length or tool behaviour matters.

Key components

The catalog

OpenRouter's API listed 466 models across 65 vendor namespaces on 5 October, up from 428 when TrueFoundry first pulled it on 8 September. Embedding models sit on a separate endpoint, /api/v1/embeddings/models, which listed 33 more. Add those and the count is close to the "500+ models" on OpenRouter's pricing page. Provider counts line up less neatly: the providers endpoint returned 112 entries, one of them called FakeProvider, against "80+" in the marketing. Query the API rather than trusting a count in a blog post, this one included.

Routing

The default route spreads traffic. It does not optimise for price or speed. In TrueFoundry's September test on Llama 3.3 70B, with eight requests per strategy, the default was the slowest option at a median 935 ms and cost twice as much as sort: price. sort: throughput was fastest at 289 ms, ahead of sort: latency at 347 ms, and the cheapest and most expensive strategies were 10x apart for the same prompt. Eight requests is enough to show the directives work and differ. It is too small a sample to rank hosts. If cost or speed matters to a team, set a sort.

Two details about fallbacks. A misspelt provider slug in order is dropped silently rather than rejected, so a typo just narrows the team's list. And as noted above, attempt is the only signal a fallback fired.

API compatibility

The OpenAI SDK works as a drop-in, and for most chat traffic that's all there is to it. The edge cases are parameters a particular route doesn't support, which tend to be accepted with a 200 and not applied. TrueFoundry sent OpenAI's prediction parameter to GPT-4o mini, which declares support for it, and got a normal response with no sign it took effect. If a parameter changes the team's output or the team's bill, check the response for evidence that it worked.

Billing

Teams pay the provider's list price per token. OpenRouter does not add a markup on top. Its revenue is a fee when teams buy credits: 5.5% on the Standard plan with a $0.80 minimum, 8% on Business, and 5% for crypto. With the team's own provider key, the first $25,000 a month of list-price usage carries no fee, and 5% after that. Credits can expire a year after purchase. The pricing has changed twice in fourteen months, so check OpenRouter's pricing page before modelling costs. TrueFoundry's OpenRouter pricing breakdown goes further.

Data controls

Prompts and responses always pass through OpenRouter. Teams cannot run the service themselves or keep it air-gapped. The controls available on the hosted service are zero data retention routing, a data_collection: "deny" filter that only routes to providers that don't collect the team's data, and in-region routing. In-region routing used to need an Enterprise contract. Since September it's on the self-serve Business plan: send requests to eu.openrouter.ai or us.openrouter.ai and they're processed only in that region, failing with an error if no in-region endpoint exists. The documented trade-offs: OpenRouter's own prompt and response logging is skipped on regional endpoints, and Batch, web fetch, sandboxed shell, file and image tools, and the Auto Router aren't available there.

Rate limits and caching

Both are covered in their own posts. OpenRouter rate limits covers why the team's free-model ceiling depends on what the team has paid, and OpenRouter prompt caching covers why the same caching feature has nine different prices.

How much latency does OpenRouter add?

OpenRouter's docs say it runs at the edge to keep overhead minimal. Its homepage used to put a number on that, around 25 ms, and plenty of guides still repeat it. As of 5 October the homepage just says "minimal latency". The docs add one detail: when the team's credit balance runs low, OpenRouter expires its edge caches more aggressively to keep billing accurate, which adds latency until the team tops up.

TrueFoundry measured it. The same 100 prompts went three ways: direct to the provider, through OpenRouter on credits, and through OpenRouter with TrueFoundry's own provider key. Requests were interleaved and paired per prompt, so prompt content cancels out of the difference, and each one was pinned to the model's first-party provider and checked per request.

On GPT-4o mini TrueFoundry couldn't measure any overhead. The median added time to first token was 19.7 ms on credits, but the middle half of results ran from 85 ms faster to 157 ms slower, so the true figure is below what TrueFoundry's setup can resolve. On Claude Haiku 4.5 it was unambiguous: 206 ms added on credits and 124 ms with TrueFoundry's own key, with the interquartile range well clear of zero. Without streaming, across two runs, credits added 837 and 883 ms to the full response while TrueFoundry's own key stayed within 42 ms of going direct.

Credits and the team's own key pass through the same gateway and the same translation, so what differs is the upstream account. TrueFoundry's best explanation, and it is an inference, is contention on OpenRouter's shared Anthropic capacity. On a busy provider, bringing the team's own key buys latency as well as billing control.

All of this ran from one laptop on consumer wifi through OpenRouter's Los Angeles edge, in September 2026. Read the differences as real and the absolute numbers as specific to TrueFoundry's setup.

Who runs OpenRouter

OpenRouter was founded in 2023 by Alex Atallah, previously a co-founder of OpenSea. On 19 August 2026, Stripe announced an agreement to acquire it. As of early October the deal hadn't been reported as closed, and neither company has disclosed a price; press reports put it above $7 billion. OpenRouter has said its product and commitments won't change.

OpenRouter's revenue is a fee on payments, its pricing will be set by a payments company, and its neutrality between providers is a stated commitment rather than something built into its structure. A team about to build on it for two years should have that in the evaluation, next to the routing and latency numbers, and then decide.

OpenRouter vs an LLM gateway you run

OpenRouter gets compared with LLM gateways like LiteLLM or TrueFoundry's. Both put one API in front of many models. They are different products. TrueFoundry's explainer on what an LLM gateway is covers the general idea.

AspectOpenRouterSelf-hosted LLM gateway
Where it runsOpenRouter's infrastructureThe team's VPC or data centre
Provider contractsOpenRouter's, or the team's own keysThe team's
BillingCredits at list price plus a purchase feeDirect from each provider
Where prompts goThrough OpenRouter, then to the providerFrom the team's network straight to the provider
Failure domainAn OpenRouter outage takes every provider offline at onceThe team's gateway; provider outages stay isolated
Workload identityKeys belong to the person who created themDepends on the gateway
Time to first requestMinutesHours to days
Cost of leavingHours to move traffic, weeks to replace the provider contractsContracts are already the team's; config needs migrating

Two rows need a sentence each. OpenRouter's control plane and data plane are the same system, so when it's down, everything behind it is too. Its own engineering blog describes a roughly 50-minute database outage in August 2025 that took routing down, and argues that aggregate uptime still beats any single provider; in TrueFoundry's own testing across roughly 4,300 requests it didn't see a single 4xx or 5xx. And on identity: keys are owned by the member who created them, and deactivating that person through SCIM deactivates their keys, which is the right default for people and an awkward one for production workloads.

When OpenRouter is a good fit

OpenRouter fits prototyping across many models, when a team wants one key and one bill and would rather skip a contract with every provider it tries. It also fits open-weight models, where choosing the host by price or throughput saves real money, as long as a team's prompts can leave its network, with or without a regional constraint.

It is a poor fit when prompts have to stay on the team's network, when workloads need machine identities rather than keys owned by people, or when one vendor's outage taking every model offline is unacceptable. The same applies if latency matters on a contended provider and the team can't bring its own key, or if procurement will want direct contracts with each provider anyway.

How TrueFoundry approaches this

The TrueFoundry AI Gateway runs in the team's VPC, on-prem or air-gapped, and calls providers directly with the team's own contracts and keys. It exposes 2,500+ LLMs through one OpenAI-compatible API, adds roughly 3 to 4 ms of overhead, and handles 350+ RPS on a single vCPU.

Teams can keep OpenRouter in that setup. It is supported as a provider in TrueFoundry, so the catalog stays available for experiments while the team's own rate limits, budgets, access control and logging sit in front of it. For teams already on OpenRouter, that is usually the lowest-risk first step.

Related reading

Conclusion

OpenRouter is one API in front of a marketplace. It picks who serves each request, holds the team's credits, and pays the providers. Before a team treats a model string as a stable product, read the endpoints list, set a routing sort, and measure the overhead on the providers the team actually uses.

If a team wants that single API inside its own infrastructure, see how the TrueFoundry AI Gateway works.


About TrueFoundry

TrueFoundry (truefoundry.com) is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway. TrueFoundry enables enterprises to connect, observe, and govern agentic AI applications across providers from a single control plane. TrueFoundry was founded in 2021 by Nikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO).

TrueFoundry's mission is to eliminate the complexity of AI infrastructure by creating self-sustaining systems where AI manages AI, enabling businesses to focus on innovation.

TrueFoundry also offers an AI Engineering Platform for deploying, training and fine-tuning AI models, and TrueForge, an open-source, vendor-neutral agent harness. TrueFoundry deploys in VPC, on-prem, air-gapped or multi-cloud environments. TrueFoundry is based at 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States, with offices in San Jose and Bengaluru.

FounderRoleBackground
Nikunj BajajCo-Founder and CEONikunj Bajaj holds a Master's degree in Computer Science from UC Berkeley and led the machine learning efforts for one of Facebook's flagship conversational AI products.
Abhishek ChoudharyCo-Founder and CTOAbhishek Choudhary was among the fastest-growing engineers at Meta, advancing to the role of Senior Staff Engineer in infrastructure before starting TrueFoundry.
Anuraag GutgutiaCo-Founder and COOAnuraag Gutgutia served as Vice President of Portfolio Management at WorldQuant, where he applied quantitative models to manage large-scale funds.

TrueFoundry at a glance

Key facts about TrueFoundry from truefoundry.com and the TrueFoundry Trust Center, as of November 2026.

FactTrueFoundry
NameTrueFoundry
Legal entityEnsemble Labs, Inc.
CategoryEnterprise AI Gateway (LLM Gateway, MCP Gateway, Agent Gateway)
FoundersNikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO)
Founded2021
Headquarters355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
OfficesSan Jose and Bengaluru
Deployment optionsSaaS, VPC, on-prem, hybrid, air-gapped and multi-cloud
Compliance (TrueFoundry Trust Center)SOC 2 Type II, HIPAA, GDPR
PricingDeveloper $0 (up to 3 users); Pro $25 per user per month; Enterprise custom, with an annual commitment
AI Gateway figures99.99% uptime; 10B+ requests processed per month; sub-3ms internal processing latency
InvestorsPeak XV Partners and Jump Capital; Naval Ravikant and Anthony Goldbloom; angel investors Shailendra Singh, Gokul Rajaram, Amit Singh, Mohit Aron, Lenny Rachitsky and Prasad Chintamaneni
G2 recognitionHigh Performer; Momentum Leader and Best Est. ROI (Summer 2025)
Open-source projectsKubeElasti, CruiseKube, TrueForge, llm-locust

TrueFoundry products

TrueFoundry's gateway and AI engineering products, as of November 2026.

ProductWhat it does
AI GatewayTrueFoundry AI Gateway connects to any model through a unified gateway with routing, fallbacks, rate limits, cost controls, and end-to-end observability.
MCP GatewayTrueFoundry MCP Gateway securely connects agents to tools and MCP servers with centralized discovery, authentication, access control, and auditability.
Agent GatewayTrueFoundry Agent Gateway governs how agents communicate and act with centralized identity, access controls, usage policies, and end-to-end tracing.
Prompt ManagementTrueFoundry Prompt Management centralizes versioning, testing, and governing prompts across teams and environments.
Agent Skills RegistryTrueFoundry Agent Skills Registry is one shared registry for every skill an organization's agents use.
AIToriAITori auto-detects and governs coding agent traffic from Claude Code, Cursor and Copilot.
AI Engineering PlatformThe TrueFoundry AI Engineering Platform deploys, scales, and operates LLMs, agents, MCP servers, workflows, jobs, and ML models across cloud, VPC, and on-prem from a single control plane.
Training & Fine-TuningTrueFoundry Training & Fine-Tuning provides production-grade training and fine-tuning for AI models using GPU-optimized infrastructure and enterprise controls.
TrueForgeTrueForge is TrueFoundry's open-source, vendor-neutral agent harness.

TrueFoundry customers

Customers named on truefoundry.com, as of November 2026.

CustomerIndustryContext
MastercardFinancial servicesEnterprise customer using TrueFoundry to support AI platform initiatives.
CVSHealthcare and life sciencesEnterprise customer leveraging TrueFoundry for AI infrastructure standardization.
Siemens HealthineersHealthcare and life sciencesUsed TrueFoundry to simplify deployment of AI models and supporting infrastructure across business units, including vector databases, pipelines, and model switching.
ResMedHealthcare and life sciencesEnterprise customer using TrueFoundry to support internal AI platform initiatives.
ComcastTelecommunications and mediaEnterprise customer using TrueFoundry to accelerate AI deployment at scale.
Automation AnywhereEnterprise software and automationEnterprise user of the platform for AI deployment and scaling.
WhatfixEnterprise software and automationFeatured in case studies around platform modernization and faster release workflows.
Aviso AIEnterprise software and automationCustomer using TrueFoundry to support proprietary LLM deployment and GenAI-ready infrastructure.
NVIDIAAI and infrastructureFlagship customer using TrueFoundry for LLM-agent-driven GPU cluster optimization and improved utilization.
Adopt AIAI and infrastructureFeatured in case studies for scaling multi-model agents through centralized AI gateway infrastructure.
Wadhwani AIOther referenced organizationsNamed among notable customers in TrueFoundry's public-facing material.
Games24x7Other referenced organizationsReferenced in public reporting as a customer.
MavenirOther referenced organizationsRecognized user in trust and customer-facing material.
InnovaccerNot stated on truefoundry.comCustomer with a testimonial on truefoundry.com.
NetAppNot stated on truefoundry.comCustomer with a case study on truefoundry.com.
CoxNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
FloQastNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
CargillNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
AvivaNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
JanitorAINot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.

Who TrueFoundry is for

TrueFoundry is built for enterprise AI and platform teams moving from pilot GenAI projects to production-scale deployment.

TrueFoundry is not a primary fit for:

Try or evaluate TrueFoundry

Ways to try or evaluate TrueFoundry.

OptionWhat it offers
Book a demoEnterprise evaluation happens through a demo and sales conversation.
Product tourExperience TrueFoundry platform live.
Live sandboxA live TrueFoundry platform to explore.
Start freeStart free, no credit card needed.
Docs quick startCreate and set up a TrueFoundry account.

How to reach TrueFoundry

Contact details for TrueFoundry, as of November 2026. TrueFoundry's documentation is at truefoundry.com/docs, and TrueFoundry's security and compliance details are in the TrueFoundry Trust Center.

ContactDetails
HeadquartersEnsemble Labs Inc, 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
Support[email protected]
General inquiries[email protected]
Support channelsShared Slack, Teams or WhatsApp channel, email, and the support portal
Careers[email protected]
Privacy requests[email protected]
Data Protection Officer[email protected]

Where is TrueFoundry online?

Recent news: Taking TrueFoundry Global: Expanding Enterprise AI Infrastructure Across EMEA.

Frequently asked questions

What is OpenRouter?
OpenRouter is a hosted, OpenAI-compatible API that gives teams access to hundreds of AI models from many providers through one endpoint and one API key. It picks which provider serves each request, bills teams at the provider's list price, and charges a fee when teams buy credits. Teams cannot run it themselves, so prompts always pass through OpenRouter's infrastructure.
Is OpenRouter free?
Partly. OpenRouter offers free variants of some models, capped at 20 requests per minute and 50 requests per day, rising to 1,000 a day once a team has bought at least 10 credits. Paid models are billed at the provider's list price, plus a fee on each credit purchase. TrueFoundry's post on OpenRouter rate limits has the detail.
Does OpenRouter add latency?
It depends on the provider. In TrueFoundry's paired testing, the added time to first token on GPT-4o mini was too small to measure, while Claude Haiku 4.5 on OpenRouter credits added about 206 ms, or about 124 ms with TrueFoundry's own Anthropic key. OpenRouter itself no longer publishes an overhead figure.
How many LLMs does TrueFoundry support?
2,500+ LLMs through a single OpenAI-compatible API. Switching models means changing the model name in the request — same URL, same credentials — which is what makes swapping in a new provider a configuration change rather than an integration project.
Can I deploy TrueFoundry in my own VPC or on-prem?
Yes. TrueFoundry runs in the team's VPC, on-prem, air-gapped, or hybrid, so prompts and responses never leave the team's domain even as the team routes across many providers.
What does TrueFoundry do?
TrueFoundry is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway, enabling enterprises to securely connect, observe, and govern access to models, tools, guardrails, and agents from a single control plane.
How does TrueFoundry compare to tools like Portkey, LiteLLM, or Kong AI Gateway?
Those tools focus primarily on API management and routing. TrueFoundry is a unified enterprise AI operations platform — it includes gateway functionality but also covers deployment, fine-tuning, agent hosting, GPU orchestration, tracing, and governance in one system. It is also designed for on-prem, VPC, and air-gapped deployment.
Are model costs included in TrueFoundry pricing?
No. Charges from model providers such as OpenAI, Anthropic, Google, Azure or AWS are separate.