Claude Batch API vs Real-Time: The 50% Cost Lever

Half your Claude bill may be for latency you are not using. A large share of production LLM work — scoring a dataset, summarizing a document backlog, generating embeddings-style labels, running an eval suite — has no human waiting on any single response. For that work, paying real-time prices is paying a premium for speed that nothing consumes. Claude’s Message Batches API exists to reclaim exactly that premium: the same models, the same quality, at 50% of the price, in exchange for accepting asynchronous delivery.

This post is about when that trade is worth taking, when it is not, and how to run batch cleanly — including through a gateway so the batch spend lands in the same cost view as everything else. The numbers below are from Anthropic’s Message Batches documentation, as of October 2026.

The trade: 50% off for asynchronous delivery

Batch is not a different model or a lower-quality path. It is the same Claude, scheduled instead of immediate.

Batch usage is charged at 50% of standard API prices. In return, the batch is processed asynchronously: the system works through it as fast as it can (most complete within an hour), and you can retrieve results once every request has finished or after 24 hours, whichever comes first. A batch that does not complete in 24 hours expires. You submit a list of requests, each with a custom_id; you poll until the batch’s processing_status is ended; then you stream the results and match them back by custom_id, because they can return in any order.

Want batch and real-time Claude spend in one cost view?

Book a demo

When batching fits — and when it doesn’t

The test is one question: is anything waiting on this response? If nothing is, batch it.

Good fits are the jobs that run on a schedule or against a backlog: evaluation and regression suites, classifying or tagging a dataset, summarizing or extracting from a document corpus, enriching records, generating synthetic data, and re-processing content after a prompt change. Bad fits are anything a person or an agent is blocked on: chat, copilots, interactive tools, and any step inside a synchronous request path. The mistake is forcing a latency-sensitive feature through batch to chase the discount — you save 50% and ship a product that makes users wait a day.

There is a middle ground worth naming: work that is tolerant of minutes but not hours. Batch is uneven here, because while most batches finish within an hour, the guarantee is only 24 hours. If a few hours of tail latency would break the workflow, treat it as real-time.

Stacking batch with prompt caching

The two biggest Claude cost levers compose. Anthropic states plainly that the prompt-caching and Message Batches discounts stack — a batched request that hits a cached prefix pays half price on a token block that is already 90% cheaper.

The catch is the cache TTL. Batched requests routinely take longer than the 5-minute default cache window, so entries written early can expire before later requests in the same batch read them. For batch, use the 1-hour cache duration on the shared prefix, keep the cacheable cache_control blocks identical across every request, and structure the batch so requests share as much static context as possible. Cache hit rates in practice range widely (Anthropic cites 30% to 98% depending on traffic shape), so, as always, measure rather than assume.

Running batch for Claude through a gateway

Here is a specific, honest limitation worth knowing. The TrueFoundry AI Gateway’s batch support runs batch jobs through the OpenAI SDK for OpenAI, Azure OpenAI, Microsoft Foundry, Bedrock, and Vertex — but not Anthropic’s first-party batch endpoint. So there are two clean ways to batch Claude.

Route one: Anthropic’s Message Batches API directly. You get the first-party 50% discount, the 100,000-request / 256 MB limits, and native caching interaction. The gateway still fronts your real-time Claude traffic and cost reporting; the batch job runs against Anthropic and you reconcile its spend.

Route two: Claude on Bedrock or Vertex, batched through the gateway. If you already run Claude via Bedrock or Vertex (common in regulated setups), you submit batch jobs through the gateway’s batch flow — upload a JSONL file, create the job with an input_file_id, endpoint, and completion_window, poll status, and fetch output_file_id. One caveat from the docs: virtual models are not supported for batch, so you pass a real catalog model identifier, and Bedrock applies a default max_tokens of 4096 for Claude Opus and Sonnet if you omit it — set it explicitly for long outputs.

Either way, the win of doing it through the gateway is the same: batch and real-time Claude spend show up in one place, attributed by team and model, so you can actually see how much the 50% lever saved.

Batch and real-time spend land in the same cost view, attributable by team and model. Source: TrueFoundry docs.

Where teams get this wrong

They batch interactive work. The 50% discount is not worth a product that makes users wait. If anything is blocked on the response, it is real-time.

They ignore the 24-hour tail. Most batches finish in an hour, but the only guarantee is 24 hours and expiry after it. Design for the tail, not the average.

They skip caching in the batch. Batch and cache discounts stack; leaving caching off on a shared-prefix batch leaves the bigger half of the saving on the table. Use the 1-hour TTL.

They expect to batch Anthropic through the gateway. The gateway batches Claude via Bedrock or Vertex; for Anthropic-direct batch, call the Message Batches API and reconcile the spend.

‍