Endpoints

Scrape

One URL into clean markdown for a RAG pipeline, an LLM prompt or an archive. Navigation, cookie walls and boilerplate stay out of it.

POST /v1/scrape

Request

Send a JSON body with a URL (or a list) and any options. The response holds data and what the call cost.

import requestsr = requests.post(    'https://api.snoopscan.com/v1/scrape',    headers={'Authorization': 'Bearer sk_YOUR_KEY'},    json={        'url': 'https://example.com/pricing',        'formats': [            'markdown',            'links'        ],        'onlyMainContent': True    },)print(r.json())

const r = await fetch('https://api.snoopscan.com/v1/scrape', {  method: 'POST',  headers: { Authorization: 'Bearer sk_YOUR_KEY', 'Content-Type': 'application/json' },  body: JSON.stringify({      "url": "https://example.com/pricing",      "formats": [          "markdown",          "links"      ],      "onlyMainContent": true  }),});console.log(await r.json());

curl -X POST https://api.snoopscan.com/v1/scrape \  -H 'Authorization: Bearer sk_YOUR_KEY' \  -H 'Content-Type: application/json' \  -d '{    "url": "https://example.com/pricing",    "formats": [        "markdown",        "links"    ],    "onlyMainContent": true}'

Returns 200 · markdown · links · 1 credit, 0 from cache.

Parameters

Field	Type	Default	What it does
`url` required	string	—	The absolute http or https URL to read.
`actions`	array of any	—	Steps to run in a browser before the page is read: click, type, scroll, wait or `captchaCheckbox`. A `captchaCheckbox` step needs `successSelector`.
`blockAssets`	boolean	true	Skip images, fonts and media when the page is rendered in a browser. Faster.
`captchaEvidence`	boolean	false	Return screenshots from before and after an automatic checkbox click.
`captchaHandling`	auto · off	auto	Click a challenge checkbox once if one appears. Set `off` to never click.
`excludeTags`	array of string	—	CSS selectors to drop before the page is read.
`formats`	array of string or object	—	What to return: `markdown`, `html`, `rawHtml`, `links`, `media`, `summary`, `screenshot`, or a `json` object carrying your schema. Ask for several in one call.
`headers`	object	—	Extra headers to send. `User-Agent` and other browser-identity headers are set for you and cannot be changed.
`includeTags`	array of string	—	CSS selectors to keep even when they sit outside the main content.
`location`	object	—	Fetch from a country, in its language, e.g. `{"country": "US"}`.
`maxAge`	integer	0	Serve a cached copy if one was taken within this many milliseconds. A cache hit costs nothing. Set `0` to force a fresh fetch.
`mobile`	boolean	false	Read the page as a phone would see it.
`onlyMainContent`	boolean	true	Return the article and drop navigation, headers, footers and cookie banners. Turn it off to keep the whole page.
`parsers`	array of string	—	Which document parsers may run, for example `pdf`.
`proxy`	object	auto	Leave on `auto` and the route is chosen for you.
`removeBase64Images`	boolean	true	Strip inline base64 images out of the returned markdown and HTML.
`storeInCache`	boolean	true	Keep this response in the cache so a later request can be served free.
`timeout`	integer	90000	Give up on this request after this many milliseconds.
`waitFor`	integer	0	Milliseconds to wait after the page loads before reading it, for content that loads late. Renders the page in a browser, and is billed as a browser page.

Response

A success returns {"success": true, "data": {…}}. data holds the formats you asked for, the page's metadata, and a cost object with what the call was charged. A failure returns {"success": false, "error": {…}} with an error code. See Errors.

Failed requests are not charged. Re-reading a page you already fetched, within maxAge, is free. See Credits for every rate.

Output formats

Crawl