Rate limits

Two separate limits: concurrency (how many scrapes run at once) and request rate (API calls per second, per key).

Embed
Preview of the card
<a href="https://scrapely.15.204.158.166.sslip.io/docs/rate-limits"><img src="https://scrapely.15.204.158.166.sslip.io/docs/_api/public/orgs/affc5b0f-d3c2-423a-8143-9d10db5736ae/spaces/981e6275-da3f-446c-82de-889f59c83b6f/pages/6dc28144-947f-4eee-8aaa-2a216c591c4f/social-card.svg?v=3e7ae33eec8b" alt="Rate limits" width="600" height="315"></a>

Two separate limits: concurrency (how many scrapes run at once) and request rate (API calls per second, per key).

Concurrency

Your concurrency limit is how many scrapes run at the same time across the whole account — sync requests, jobs and structured calls share it. It grows with lifetime credits purchased (see Credits); GET /v1/account shows your limit and what is in flight.

  • A sync request with no free slot fails immediately: 429 CONCURRENCY_LIMIT with Retry-After: 1.

  • An async job never fails for concurrency: it waits in your queue.

Request rate

Separately, each API key may make a bounded number of calls per second (50 by default), counting every endpoint — including cheap ones like polling a job. Exceeding it is 429 RATE_LIMITED.

Header

Meaning

X-RateLimit-Limit

Calls allowed per window.

X-RateLimit-Remaining

Calls left in the current window.

X-RateLimit-Reset

Unix time (seconds) when a call frees up.

Retry-After

On a 429: seconds to wait.

Stop polling: The usual way to hit the rate limit is polling job status in a loop. Pass callback_url instead and let us tell you.

Updated