Claude’s Message Batches API: Async Processing and the 24-Hour Window

#ai #machinelearning #llm #programming

The Message Batches API takes a list of requests you would otherwise send one at a time, processes them asynchronously, and charges less for the privilege. The cost is latency: you give up any guarantee about when an individual request completes, in exchange for a documented ceiling on when the whole batch does.

What the trade is

Anthropic documents batch processing as returning results within 24 hours, at a 50% discount on both input and output tokens compared with the equivalent standard API calls. Both figures are published on Anthropic’s batch processing documentation and its pricing page, and the discount is applied automatically — there is no flag to set.

The 24 hours is a ceiling, not an estimate. Batches frequently finish far sooner, and there is nothing in the API that lets you ask for faster. That is the whole decision: if any human is waiting on an individual result, this is the wrong endpoint. If the work is classification over a backlog, evaluation runs, bulk summarisation, enrichment of a table, or anything else where the deadline is “tomorrow” rather than “now”, halving the bill for no code complexity beyond polling is close to free.

What batching does not give you is worth stating as plainly as what it does. There is no ordering guarantee, no partial-results stream, no way to prioritise one request inside a batch, and no progress signal finer than the counts in request_counts. Each request is independent — a batch is not a conversation, and entries cannot refer to each other. And a batched request is a normal request in every other respect, including that it can fail on its own terms: a bad schema or an oversized prompt fails that line, not the submission.

The discount and the completion window are pricing and service commitments, and both are the kind of figure that changes. Read them off Anthropic’s pricing and batch pages rather than from here before you build a cost model on them.

Submitting a batch

A batch is a POST to /v1/messages/batches with a requests array. Each entry has a custom_id you choose and a params object that is exactly a Messages API request body — same model, same required max_tokens, same system, tools and messages:

POST https://api.anthropic.com/v1/messages/batches
anthropic-version: 2023-06-01
content-type: application/json

{
  "requests": [
    {
      "custom_id": "ticket-90214",
      "params": {
        "model": "claude-sonnet-4-20250514",
        "max_tokens": 512,
        "system": "Classify the ticket. Reply with one word.",
        "messages": [{ "role": "user", "content": "Card declined at checkout..." }]
      }
    },
    {
      "custom_id": "ticket-90215",
      "params": {
        "model": "claude-sonnet-4-20250514",
        "max_tokens": 512,
        "system": "Classify the ticket. Reply with one word.",
        "messages": [{ "role": "user", "content": "How do I export my data?" }]
      }
    }
  ]
}

custom_id is load-bearing and it is the field people under-think. Results come back unordered, so the ID is your only join key back to your own records. It must be unique within the batch. Use your primary key, not the array index — an index survives exactly until somebody filters the input list.

Nothing else about the request changes. A batched request can use tools, images, extended thinking and prompt caching; because entries within a batch are processed close together, a shared cached prefix across a batch is worth setting up rather than assuming caching and batching are alternatives. See cache_control and prompt caching.

Polling and processing status

The response to submission is a Message Batch object, immediately, with an id beginning msgbatch_. You then poll GET /v1/messages/batches/{id} until it is done:

{
  "id": "msgbatch_013Zva...",
  "type": "message_batch",
  "processing_status": "in_progress",
  "request_counts": {
    "processing": 2,
    "succeeded": 0,
    "errored": 0,
    "canceled": 0,
    "expired": 0
  },
  "ended_at": null,
  "created_at": "2026-08-11T09:14:02Z",
  "expires_at": "2026-08-12T09:14:02Z",
  "results_url": null
}

processing_status takes three values: in_progress, canceling and ended. Note what ended does not mean — it means the batch is finished being processed, not that everything in it succeeded. The counts in request_counts are where success lives. A batch of 10,000 with 10,000 in errored is ended.

Poll on an interval measured in tens of seconds at most, with backoff; there is no webhook obligation to hold a connection open, and hammering the status endpoint every second for a job with a 24-hour window buys nothing.

Reading results, including the failures

When processing_status reaches ended, results_url is populated. It streams .jsonl — one JSON object per line, one line per request, in no guaranteed order. Each line carries your custom_id and a result whose type is one of four values:

{"custom_id":"ticket-90215","result":{"type":"succeeded","message":{
   "id":"msg_01...","content":[{"type":"text","text":"billing"}],
   "stop_reason":"end_turn","usage":{"input_tokens":41,"output_tokens":3}}}}

{"custom_id":"ticket-90214","result":{"type":"errored","error":{
   "type":"invalid_request_error","message":"..."}}}

succeeded — a normal Message object under message, complete with its own stop_reason and usage. Check stop_reason here exactly as you would on a live call; a batched response can still come back truncated at max_tokens.
errored — a per-request error object in the same shape as a synchronous API error. This is the one that matters most for correctness: a malformed request inside a batch does not fail the batch, it fails one line, and code that iterates results assuming result.message exists will throw on it.
canceled — the request had not started when you cancelled the batch.
expired — the request was still unprocessed when the 24-hour window closed. You are not charged for these, and they are the ones to resubmit.

Results remain retrievable for a documented retention period after the batch ends — long enough that you do not need to consume them immediately, short enough that you should not treat the results URL as storage. Write the lines somewhere durable as you stream them.

Limits and the expiry rule

Anthropic documents a per-batch ceiling of 100,000 requests or 256 MB of total request size, whichever is hit first. That second limit binds earlier than people expect on multimodal work: base64 images in params count towards it, so a batch of a few thousand image-bearing requests can exceed 256 MB long before it approaches 100,000 entries. Split by size, not by count.

The expiry rule is the operational one to design around. The window runs from creation, so anything not processed 24 hours later comes back as expired rather than being carried over. A resubmission loop that reads the results file, filters for expired and errored, and builds a fresh batch from those custom_ids is about fifteen lines and is the difference between a pipeline that finishes and one that quietly loses a slice of its rows every night.

Cancellation is best-effort rather than immediate. Posting to /v1/messages/batches/{id}/cancel moves the batch to canceling, and requests already in flight are allowed to complete — so a cancelled batch still produces a results file, mixing succeeded and canceled entries. You are charged for what completed. There is no way to cancel one request within a batch; granularity is the batch.

Two smaller things that save an afternoon. Batches do not consume the same rate limits as synchronous traffic in the same way, which is much of the appeal — a backlog that would take hours to push through per-minute token limits goes in as one submission. And there is no idempotency on submission: posting the same requests array twice creates two batches and bills both. If your submitter can retry on a network timeout, record the returned msgbatch_ ID before doing anything else, and check for an existing batch before resubmitting a job that may already be in flight.

Batch endpoints are one of the places provider APIs diverge most: different submission shapes, different status enums, different result encodings, and different windows. If you are batching against more than one vendor, the reconciliation logic — join on your own ID, classify per-row outcomes, resubmit the failures — is worth writing once against a normalised result type rather than twice against two vendor schemas. That normalisation is the part a gateway such as Multigrid is for; the 24-hour window and the discount remain the provider’s.