One user, 200 API calls: rate limits when agents run in parallel

An agent splits an account report into parallel reads. Other workers are already using the same API connection. The new requests consume the account's remaining quota, while slow requests hold its in-flight slots.

The title's 200 calls describes an example workload, not an industry measurement. When workers share an API account, their limits need to account for each other's requests.

Rate and concurrency are different limits

A request-rate limit bounds work admitted over a time interval. A concurrency limit bounds requests active at the same time. Slow operations can exhaust concurrency even when the number of new requests per second appears modest.

Stripe's rate-limit documentation explicitly distinguishes these controls for its API. Its exact limits and response details are specific to Stripe. For another provider, inspect the documented quota dimensions, error signals and retry instructions rather than assuming a common implementation.

Also identify the bucket: user, account, organization, endpoint or credential. Giving each agent a local limiter does not coordinate a quota shared across an entire account.

Reserve capacity before dispatch

Put shared admission control at the layer that owns the API connection or shared quota. Reserve an in-flight slot before sending a request and release it when that attempt ends. Apply a separate rate budget where the provider requires one.

In a distributed deployment, workers need a coordination mechanism with clear failure behavior. A process-local semaphore only controls its own process. Consider what happens when a worker exits while holding a lease and how capacity becomes available again.

Keep the queue bounded. If admission would exceed the task's deadline, return a clear partial or deferred result rather than allowing an invisible backlog to grow indefinitely. Preserve fairness so one large report cannot starve interactive requests.

Bound the agent as well as the API

Give a task a request budget, total deadline and limit on parallel work. Count retries and follow-up detail reads when they consume the same provider quota. Otherwise a failed request can spawn more work precisely when the shared system is overloaded.

Honor documented retry guidance, including server-provided delays where applicable. Backoff should reduce pressure; immediate synchronized retries across workers can recreate the original overload.

If tasks may create paid resources or perform billable processing, add an explicit application spending budget. Request limits and currency limits measure different things, and neither can stand in for the other.

Test a shared account, not isolated workers

Run controlled tests with several agents using the same synthetic account. Include slow requests, cancellation, a worker crash, a quota response and a task that uses more calls than expected. Observe queue depth, active requests, wait time and denied admissions.

Distinguish time spent waiting for local capacity from time spent on the provider's network request. Those measurements help identify whether the right improvement is caching, batching, scheduling or a provider limit increase.