Claude API Batch Request Processing Guide

If you need to run Claude over thousands of prompts — summarizing documents, classifying support tickets, extracting data from PDFs — sending requests one at a time in a loop will either take forever or get you rate-limited. Batch request processing means sending many Claude API calls concurrently (or in a dedicated batch queue) while respecting rate limits, handling partial failures, and reassembling results in the right order.

This guide walks through practical patterns for batch processing with the Claude API: controlling concurrency, retrying transient errors, tracking progress, and keeping costs predictable. The same patterns apply whether you're calling Anthropic's API directly or going through a proxy like SubToAPI.

Why naive loops don't work

A simple for loop that awaits each request sequentially is correct but slow — if each call takes 2-3 seconds, 10,000 items takes 6-8 hours. The opposite extreme, firing all requests at once with Promise.all, will hit rate limits (HTTP 429) almost immediately and waste retries on requests that were never going to succeed.

The right approach sits in the middle: a bounded concurrency pool that keeps N requests in flight at all times, refilling the pool as responses come back.

Concurrency-controlled batch processing

Here's a Node.js pattern using a simple worker pool, no external queue library required:

async function processBatch(items, worker, concurrency = 5) {
  const results = new Array(items.length);
  let index = 0;

  async function runWorker() {
    while (index < items.length) {
      const current = index++;
      try {
        results[current] = await worker(items[current]);
      } catch (err) {
        results[current] = { error: err.message };
      }
    }
  }

  const workers = Array.from({ length: concurrency }, runWorker);
  await Promise.all(workers);
  return results;
}

Then define the worker function that calls Claude:

async function summarize(text) {
  const res = await fetch("https://api.subtoapi.app/v1/messages", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
    },
    body: JSON.stringify({
      model: "claude-sonnet-4",
      max_tokens: 300,
      messages: [{ role: "user", content: `Summarize in 2 sentences:\n\n${text}` }],
    }),
  });

  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  const data = await res.json();
  return data.content[0].text;
}

const results = await processBatch(documents, summarize, 5);

Start with a concurrency of 5-10 and increase it only if you're not seeing 429s. Each account has a different rate ceiling depending on plan and model, so test before scaling up.

Handling rate limits and retries

Even with bounded concurrency, you'll occasionally hit a 429 or a transient 5xx. Wrap the actual API call in a retry function with exponential backoff:

async function callWithRetry(fn, maxRetries = 4) {
  let attempt = 0;
  while (true) {
    try {
      return await fn();
    } catch (err) {
      attempt++;
      if (attempt > maxRetries) throw err;
      const delay = Math.min(1000 * 2 ** attempt, 15000);
      await new Promise((r) => setTimeout(r, delay));
    }
  }
}

Only retry on 429 and 5xx responses — don't retry 400-level errors caused by bad input, since they'll fail the same way every time and just burn your retry budget.

Chunking large jobs

For jobs with tens of thousands of items, don't hold everything in memory. Process in chunks and persist progress so a crash doesn't force you to restart from zero:

const CHUNK_SIZE = 200;

for (let i = 0; i < items.length; i += CHUNK_SIZE) {
  const chunk = items.slice(i, i + CHUNK_SIZE);
  const chunkResults = await processBatch(chunk, summarize, 8);
  await saveResults(chunkResults, i);
  console.log(`Processed ${i + chunk.length}/${items.length}`);
}

Writing results to disk or a database after each chunk means you can resume from the last checkpoint instead of reprocessing items that already succeeded.

Tracking cost and token usage

Batch jobs are where token costs add up fast, so track usage per request rather than waiting for a surprise bill. Claude's response includes usage metadata:

const data = await res.json();
console.log(data.usage.input_tokens, data.usage.output_tokens);

Sum these across the batch and log them alongside each result. If you're running jobs across a team, a dashboard that shows usage per API key makes it much easier to catch a runaway job before it finishes — this is one of the reasons teams route batch workloads through SubToAPI, which gives every application its own sub_live_ key with usage visible in one dashboard instead of digging through raw logs.

Picking the right concurrency and model

A few practical rules of thumb:

• Use a smaller/faster model for simple tasks (classification, short extraction) and reserve the larger model for genuinely hard reasoning. This cuts both latency and cost across a large batch.
• Set max_tokens conservatively. A batch of 10,000 requests with an unnecessarily high max_tokens cap can silently run much longer and cost more than intended.
• Keep prompts short and templated. Reusing the same system prompt structure across all items make outputs more consistent and easier to parse downstream.
• Log failures separately from successes so you can re-run just the failed subset instead of the whole batch.

If you want a working starting point, the quickstart covers authentication and your first request, and the messages endpoint docs detail every parameter you can tune per request, including max_tokens, temperature, and stop_sequences.

When to use streaming instead

Batch processing is for throughput over many independent items. If you're building a single long-running response — a long document generation or an interactive chat — you want streaming instead, which returns tokens incrementally rather than waiting for the full completion. The two patterns aren't mutually exclusive: you can run a batch of streaming calls if you need partial results per item as soon as they're ready, though for most offline batch jobs non-streaming requests are simpler to manage.

Questions

What's a safe concurrency level for batch Claude API requests? Start at 5-10 concurrent requests and watch for 429 responses. If you see none after a few minutes, increase gradually. Rate limits vary by plan and model, so there's no universal number — test against your own account.

Should I retry every failed request in a batch? No. Retry 429 and 5xx errors with exponential backoff, but don't retry 400-level errors — these indicate a problem with the request itself (bad input, missing fields) that won't resolve on retry.

How do I keep batch job costs under control? Track input_tokens and output_tokens from each response, cap max_tokens per request, and use smaller models for simple tasks. Tools like SubToAPI's dashboard make it easy to monitor usage across an entire batch job per API key in real time.