What's new
Explainer14 Nov 20265 min read

Your request limit may not be the limit you hit

Under the assumed limits here, raising an answer from 1,000 to 2,000 tokens cuts the steady request ceiling from thirty to fifteen per minute, and adding workers cannot lift an output-token limit.

The largest request count on a limits page may not be the one that governs your workload. A queue of jobs and a generous allowance make it tempting to launch enough workers to use it, until requests begin failing or waiting. The token allowances are worth checking before concurrency is touched.

Anthropic's rate-limit documentation, read on 14 November 2026, distinguishes requests per minute, input tokens per minute and output tokens per minute. It also warns that bursts can exceed limits. We use hypothetical allowances below so the calculation does not depend on a changing account tier.

Each allowance converts to a number of jobs

Assume sixty requests per minute, 120,000 input tokens per minute and 30,000 output tokens per minute. Dividing each token allowance by what one request uses gives a job count for each, and the smallest of them is the simple steady-state ceiling.

Hypothetical limits: 60 RPM, 120,000 input tokens/minute and 30,000 output tokens/minute. No provider tier is being quoted.
Assumed input/output per requestRequest ceilingInput ceilingOutput ceilingNarrowest, requests/minute
1000/2006012015060
4000/100060303030
4000/200060301515

A small request hits the request-count ceiling first. The middle workload reaches thirty jobs through both token allowances. The longer-output workload reaches fifteen through output, even though its input could support thirty. So the arithmetic says which resource deserves attention before any queue tuning.

These are planning ceilings, not observed delivery rates. Real requests vary in size and arrive unevenly, the provider may estimate usage during processing, and the account may share capacity with another application. Record those conditions when comparing a prediction with a measured run.

Concurrency and throughput are different measures

Concurrency counts requests in flight, and throughput counts work completed over time. To plan a trial, estimate how many requests must overlap to reach the desired completion rate at the observed request duration, then increase cautiously and measure accepted completions rather than assuming more simultaneous calls must help.

If the queue grows, arrival rate and completion rate should be recorded separately, since a service completing steadily can still fall behind a larger incoming workload. A backlog is an application-level observation, not proof that the underlying model slowed down.

Queue waiting belongs outside the provider-response timer and inside the user's total wait. That split separates time spent waiting for permission to send from time after submission.

Usage fields mean what the provider documents

Raw usage categories are worth saving privately, with a clearly labelled summary derived from them. Cached input, new input and generated output can be counted differently for billing and for limits, so check the exact provider and model rules before subtracting anything from a token allowance.

The table deliberately assumes no caching. Adding it means recomputing the relevant allowance from documented accounting and measured hit behaviour. A billing discount and a throughput benefit are separate claims, and seeing the word “cache” in the configuration supports neither.

When a limit response gives a retry delay, respect it. Releasing the whole waiting queue at the same instant can create another burst. A bounded scheduling policy, with the number of attempts visible, stops retries masquerading as new demand.

A run only means something under a stated load

A useful run record includes the date the account allowance was checked, workload token distribution, concurrency, queue delay, accepted completions and limit responses. Credentials and account identifiers stay out of any public evidence, and synthetic workloads or aggregate counts are enough to explain a result.

The aim is a queue that finishes the required work within its deadline. Find the narrowest allowance first, then measure what the whole workflow sustains under it.