Documentation

One endpoint

POST /v1/scrape — a URL in, formatted content out. Everything else is account management.

The request

curl

curl https://api.webfetch.run/v1/scrape \
  -H "Authorization: Bearer $SCRAPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown"],
    "egress": "residential",
    "persist": true
  }'

Field	Type	What it does
url	string	Required. The page to fetch. Must be public http(s).
formats	string[]	Defaults to ["markdown"]. Names are case-insensitive; an unknown name is a 400 that lists the valid ones.
egress	"direct" \| "datacenter" \| "residential"	Where the request leaves from. Defaults to direct, the cheapest tier.
onlyMainContent	boolean	Defaults to true: narrows html and markdown to the main content. Set false for the whole DOM.
waitForSelector	string	Wait for this CSS selector before capturing.
persist	boolean	Store the capture and return a link that renders it from our origin. Off by default.
rotations	number	How many exits to try on a wall. Default 3, clamped 0-5.
parallelProbes	number	How many exits to try concurrently. Default 3, clamped 1-5.
solve	boolean	Whether a CAPTCHA may be attempted. Default true.
solveTimeoutMs	number	Cap on one solve. Default 20000, clamped 5000-120000.
timeout	number	Overall budget for the request. Default 120000, clamped 5000-300000.
session	object	Replay a cleared session: the session.cookies (and egressPort) from a previous response.
attributes	{selector,attribute}[]	For the attributes format. Up to 20.
summaryPrompt	string	Instruction for the summary format.
jsonOptions	{schema}	JSON Schema for the json format.

Formats

Ask for any combination. You get back only what you asked for.

Format	Returns	What it is
markdown	string	The page as markdown. The default when no format is given.
html	string	Cleaned HTML. Narrowed by onlyMainContent, which defaults to true.
rawHtml	string	The document exactly as received, never narrowed.
rawBase64	string	The document bytes as base64. UTF-8 safe.
text	string	Visible text only, no markup.
screenshot	string	Base64 PNG of the rendered page.
links	string[]	Absolute URLs found on the page.
images	{src,alt,...}[]	Absolute image URLs with alt text and dimensions when present.
metadata	object	The page's head tags: title, description, og:*, canonical, language.
branding	object	Colours, fonts and logo, extracted deterministically.
attributes	object	Named attributes per CSS selector. Send `attributes: [{selector, attribute}]`.
summary	string	A short abstract, generated by a model.
json	object	Structured extraction. Send `jsonOptions: { schema }`.

The response

200 OK

{
  "success": true,
  "data": {
    "markdown": "# Example Domain\n\nThis domain is for use...",
    "metadata": {
      "url": "https://example.com",
      "finalUrl": "https://example.com/",
      "title": "Example Domain",
      "status": 200,
      "verdict": "ok",
      "tier": "browser/residential",
      "attempts": 1,
      "bytes": 1258,
      "session": { "cookies": ["..."], "egressPort": 10006 }
    }
  },
  "credits": { "charged": 10, "remaining": 990 }
}

Keep metadata.session and send it back as session on the next request. A cleared session is bound to the exit that earned it, and reusing it turned repeat queries from 38 s into 6-8 s.

Errors

A failure is always success: false with a machine-readable code, and always costs 0 credits.

Code	HTTP	Meaning
invalid_request	400	The body did not validate — a missing url, an unknown format, a bad selector shape.
target_not_allowed	400	The URL is private, internal, or not a public http(s) address.
egress_unavailable	400	You asked for an egress tier this deployment has not configured. Refunded in full.
insufficient_credits	402	The balance cannot cover the maximum this request could cost. Nothing ran.
blocked	200	The site returned a wall instead of content. `success: false`, 0 credits charged.
timeout	504	The request exceeded its `timeout` budget. 0 credits charged.
fleet_unavailable	503	Our own renderer could not be reached. Our fault, refunded in full.

Keys

Create a key in the console. The plaintext is shown once; we store only a SHA-256 hash. Send it as Authorization: Bearer sk_live_....

Manage keys Pricing