Scraping API

GET /v1/scrape fetches one page synchronously, with proxies, retries and optional rendering, and returns it.

Embed
Preview of the card
<a href="https://scrapely.15.204.158.166.sslip.io/docs/scrape-api"><img src="https://scrapely.15.204.158.166.sslip.io/docs/_api/public/orgs/affc5b0f-d3c2-423a-8143-9d10db5736ae/spaces/981e6275-da3f-446c-82de-889f59c83b6f/pages/909875d7-ad0e-4004-9c83-38bf2aaa77bf/social-card.svg?v=3e7ae33eec8b" alt="Scraping API" width="600" height="315"></a>

GET /v1/scrape fetches one page synchronously, with proxies, retries and optional rendering, and returns it.

Endpoint

GET  https://api.scrapely.dev/v1/scrape?url=…
POST https://api.scrapely.dev/v1/scrape        (same parameters as JSON or a form)

The call blocks until the page is ready or we give up — up to 70 seconds while we retry through blocks. It occupies one concurrency slot while it runs; with none free it fails fast with 429 CONCURRENCY_LIMIT (async jobs queue instead).

Parameters

Names match ScraperAPI, so existing integrations work by changing only the host. Booleans accept true/false or 1/0.

Parameter

Type

Default

Description

url (required)

string

—

The page to scrape: an absolute http(s) URL, URL-encoded when sent in a query string. Private, loopback and link-local hosts are refused.

render

boolean

false

Load the page in a headless browser and execute its JavaScript before returning the DOM. Cost: 10 credits.

premium

boolean

false

Route through the residential proxy pool. Use for sites that block datacenter IPs. Cost: 10 credits (25 with render).

ultra_premium

boolean

false

Our hardest-target pipeline: premium residential IPs, browser fingerprinting and CAPTCHA handling. Includes rendering. Cannot be combined with premium. Cost: 30 credits.

country_code

enum

—

Geotarget the request from this country (lowercase ISO 3166-1 alpha-2, e.g. us, de, jp).

device_type

desktop, mobile

desktop

Send a desktop or mobile user agent and viewport.

session_number

integer

—

Reuse the same proxy IP across requests that share this number (sticky session, ~15 minutes idle timeout).

keep_headers

boolean

false

Forward the headers you send us (Cookie, Accept-Language, User-Agent, …) to the target instead of ours. Your API key, Host and proxy headers are never forwarded. Sync endpoint only.

output_format

html, json, markdown, text

html

Return the page as raw HTML, cleaned Markdown, plain text, or JSON (title, meta, links, text).

autoparse

boolean

false

For supported domains, return structured JSON parsed from the page instead of HTML.

wait_for_selector

string

—

With render=true: wait until this CSS selector appears before capturing the page.

raw

boolean

false

Return the target's body directly (as ScraperAPI does) instead of the JSON envelope. Metering headers are still set.

Response

A JSON envelope by default. content is a string for html, markdown and text, and an object for json and autoparse.

200 OK

{
  "url": "https://example.com/products/42",
  "final_url": "https://example.com/products/42",
  "status_code": 200,
  "credits": 10,
  "duration_ms": 2140,
  "content_type": "text/html; charset=utf-8",
  "content": "<!doctype html>…"
}

Headers

X-Request-Id: req_3c9f1e…
X-Scrapely-Credits: 10
X-Scrapely-Balance: 141270
X-Scrapely-Target-Status: 200
X-RateLimit-Limit: 50
X-RateLimit-Remaining: 49
X-RateLimit-Reset: 1792000001

Field

Description

status_code

The target's status. 200, or 404/410 — a definitive "not found" is a real answer and is billed.

credits

What this call cost. Also in X-Scrapely-Credits; your remaining balance is in X-Scrapely-Balance.

final_url

After redirects.

content_type

Of content: the target's type for html, or ours after conversion.

Raw mode

With raw=true the response IS the page: the target's body, Content-Type and status (200 or 404). Metering headers are still set. This is exactly ScraperAPI's response shape.

curl "https://api.scrapely.dev/v1/scrape?api_key=YOUR_API_KEY&raw=true&url=https%3A%2F%2Fexample.com" -o page.html

Output formats

output_format

You get

html (default)

The page as served (or as rendered, with render=true).

markdown

Main content as Markdown: headings, links (absolute), lists, emphasis. Scripts, styles and navigation chrome removed. Ideal for LLM pipelines.

text

Visible text only, whitespace-normalised.

json

{ title, description, canonical, lang, headings[], links[], text }.

With autoparse=true, a URL that one of our live structured endpoints understands comes back as that endpoint's typed JSON, at the scrape's normal price. The structured endpoints are coming soon, so for now autoparse=true returns the json format. For Amazon products, use the Amazon Product API.

Geotargeting {#country_code}

country_code takes a lowercase ISO 3166-1 alpha-2 code. 64 countries are available:

us ca mx br ar cl co pe gb ie fr de nl be lu ch at es pt it gr se no dk fi is pl cz sk hu ro bg hr si ee lv lt ua tr il ae sa eg za ng ke ma in pk bd lk th vn my sg id ph cn hk tw kr jp au nz

Sticky sessions

Requests that share a session_number reuse the same proxy IP, for flows that must look like one visitor (pagination behind a login, multi-step checkouts). A session idles out after about 15 minutes. Any integer works; use one per logical visitor.

What you pay

Credits are reserved when the request starts and refunded if it fails. You pay only for 2xx, 404 and 410 answers. Blocks (403), rate limits (429), server errors and timeouts that survive our retries are free. See Credits & billing.

Updated