Say your Claude bill has a long tail of non-interactive work: overnight classification, dataset labeling, monthly content moderation sweeps, eval runs. While those calls go through the synchronous Messages API, you leave about half the money on the table. The Batch API exists for exactly this. You submit a job, wait up to 24 hours, and pay 50% less on both input and output tokens.
What the Batch API actually is
The Batch API is an asynchronous job queue. Requests do not go out one by one with a wait for each answer. You upload a JSONL file where every line holds a complete Messages API request, Anthropic works through the whole batch under a 24-hour SLA, and you download a JSONL file of responses keyed to the custom_id on each line.
The economics are simple:
- Input tokens: 50% off.
- Output tokens: 50% off.
- Stacks with prompt caching. You can layer the cache discount on top of the batch discount for the cached part of your input.
- One batch takes up to 100,000 requests.
- A 24-hour SLA. Most batches finish within minutes to a few hours, but you cannot plan around that speed.
Nothing runs Claude cheaper at scale. Full stop.
Which workloads suit Batch
Batch fits when none of your individual requests is time-sensitive. Concretely:
- Overnight classification of millions of records. You tag support tickets, route leads, score documents for relevance. Submit at 18:00, collect the results by morning.
- Dataset labeling and synthetic data. You build eval sets, generate training pairs, and enrich RAG corpora with summaries, titles, or extracted entities.
- Monthly content moderation sweeps. User-generated content gets rescanned against an updated policy. The job runs once a month, latency does not matter, and the volume is very large.
- Eval runs. You push a new prompt or model through the full regression set of thousands of test cases.
- RAG corpus enrichment. You precompute summaries, FAQ extractions, and chunk-level classes for the knowledge base. Build once, serve cheaply afterwards.
- Bulk semantic work in place of embeddings. You would normally reach for embeddings plus a classifier, but what you need is a language-model judgment on each item. Batch makes the LLM-per-item route affordable.
When not to use Batch
It matters just as much to know where Batch fails:
- Interactive UX. Anything a user is waiting on: chat, code completion, in-product agents. A 24-hour SLA is fatal here.
- Urgent flows. Fraud decisions, live moderation, alerting. These need an answer now.
- Tiny jobs. With 50 requests, you spend more effort on uploading, polling, and joining the results than the saved pennies are worth. Batch starts to pay off in the thousands.
- Chains with tight dependencies between requests. If request N+1 needs the output of request N, you need a synchronous loop, not a batch.
How the price works out: a worked example
Take a realistic bulk classification job: 100,000 Sonnet 4.6 calls, with 1,000 input tokens and 500 output tokens each. That adds up to 100M input tokens and 50M output tokens.
At synchronous rates you know the total from your monthly invoice. Through the Batch API the same workload costs exactly half:
- Synchronous: input cost + output cost =
X. - Batch:
X / 2.
At this scale, that 50% often decides whether you run the job once a quarter or every night. If you also cache a stable system prompt across all 100,000 requests, the cache discount stacks on the cached part, and the effective input price drops further.
Code: building the batch JSONL
Every line of the input file is a self-contained request that carries your custom_id. Through codexkey, point the client at https://api.codexkey.online/v1. The Batch API is passed through transparently.
import json
from anthropic import Anthropic
client = Anthropic(
base_url="https://api.codexkey.online/v1",
api_key="cxa_your_key_here",
)
# 1. Build the JSONL request file
records = load_records_to_classify() # your 100k items
with open("batch.jsonl", "w") as f:
for r in records:
line = {
"custom_id": f"record-{r['id']}",
"params": {
"model": "claude-sonnet-5",
"max_tokens": 256,
"system": [
{
"type": "text",
"text": CLASSIFIER_SYSTEM_PROMPT,
"cache_control": {"type": "ephemeral"},
}
],
"messages": [
{"role": "user", "content": r["text"]},
],
},
}
f.write(json.dumps(line) + "\n")
# 2. Submit the batch
batch = client.messages.batches.create(
requests=[json.loads(line) for line in open("batch.jsonl")]
)
print("submitted:", batch.id)
Polling and reading results
Polling is simple. Check processing_status at a sensible interval: every 60 seconds is plenty, and for most jobs every 5 minutes is enough. Skip the hot loop.
import time
while True:
b = client.messages.batches.retrieve(batch.id)
if b.processing_status == "ended":
break
time.sleep(60)
# Stream results line-by-line
for result in client.messages.batches.results(batch.id):
cid = result.custom_id
if result.result.type == "succeeded":
message = result.result.message
save_classification(cid, message.content[0].text)
elif result.result.type == "errored":
log_error(cid, result.result.error)
elif result.result.type == "expired":
# request didn't complete within 24h — requeue
requeue(cid)
Errors are handled per line
One detail to remember: the Batch API reports errors per request, not per batch. A handful of failed lines do not spoil the rest. Your processing code has to handle three terminal states for each custom_id:
succeeded: a normal response. Store it and move on.errored: a model or validation error. Log it, optionally retry as a one-off synchronous call, and continue.expired: the SLA ran out before this request was served. Put it into the next batch.
Treat the result file as a stream. Do not load 100k responses into memory.
Add prompt caching: the discounts stack
Batch and prompt caching are independent discounts. If your 100,000 requests share a long system prompt or document context, mark it with cache_control. The cached part then gets the cache discount on top of the 50% batch discount. When an agent-style classifier carries a 4k-token system prompt, this is the biggest lever after moving to batch in the first place.
Operational note when using codexkey
codexkey passes the Batch API through to Anthropic transparently: the same endpoints, the same JSONL format, the same SDK. One point is specific to a pay-per-token gateway. A batch is charged at submission, so top up enough balance to cover the whole job before you call batches.create. The dashboard shows an estimated cost per batch based on the input file. Budget against that figure and add a margin for output tokens.
Bottom line
If a job does not have to answer in seconds, it does not belong on the synchronous API. Move overnight jobs, dataset processing, monthly sweeps, and eval runs to Batch, add prompt caching on the stable parts, and keep the synchronous API for what a user is actually waiting for.