Endpoints

Scrape

One URL into clean markdown for a RAG pipeline, an LLM prompt or an archive. Navigation, cookie walls and boilerplate stay out of it.

POST /v1/scrape

Request

Every tool takes the same shape: a URL or a list, a few options, and back comes data plus what the call cost.

import requestsr = requests.post(    'https://api.snoopscan.com/v1/scrape',    headers={'Authorization': 'Bearer sk_YOUR_KEY'},    json={        'url': 'https://example.com/pricing',        'formats': [            'markdown',            'links'        ],        'onlyMainContent': True    },)print(r.json())

const r = await fetch('https://api.snoopscan.com/v1/scrape', {  method: 'POST',  headers: { Authorization: 'Bearer sk_YOUR_KEY', 'Content-Type': 'application/json' },  body: JSON.stringify({      "url": "https://example.com/pricing",      "formats": [          "markdown",          "links"      ],      "onlyMainContent": true  }),});console.log(await r.json());

curl -X POST https://api.snoopscan.com/v1/scrape \  -H 'Authorization: Bearer sk_YOUR_KEY' \  -H 'Content-Type: application/json' \  -d '{    "url": "https://example.com/pricing",    "formats": [        "markdown",        "links"    ],    "onlyMainContent": true}'

Returns 200 · markdown · links · 1 credit, 0 from cache.

Parameters

Read from the engine itself, so this table is the request it actually validates.

Field	Type	Default	What it does
`url` required	string	—	The absolute http or https URL to read.
`actions`	array of any	—	Click, type, scroll or wait in a real browser before the page is read.
`blockAssets`	boolean	true	Skip images, fonts and media when a browser is used. Faster, and cheaper on bandwidth.
`excludeTags`	array of string	—	CSS selectors to drop before the page is read.
`formats`	array of string or object	—	What to return: `markdown`, `html`, `rawHtml`, `links`, `media`, `summary`, `screenshot`, or a `json` object carrying your schema. Ask for several in one call.
`headers`	object	—	Extra request headers. Headers that identify the client are set for you and cannot be overridden.
`includeTags`	array of string	—	CSS selectors to keep even when they sit outside the main content.
`location`	object	—	Fetch from a particular country, with that country's language.
`maxAge`	integer	172800000	Serve a cached copy if one was taken within this many milliseconds. A cache hit costs nothing. Set `0` to force a fresh fetch.
`mobile`	boolean	false	Read the page as a phone would see it.
`onlyMainContent`	boolean	true	Return the article and drop navigation, headers, footers and cookie banners. Turn it off to keep the whole page.
`parsers`	array of string	—	Which document parsers may run, for example `pdf`.
`proxy`	object	auto	Leave on `auto` and the route is chosen for you.
`removeBase64Images`	boolean	true	Strip inline base64 images out of the returned markdown and HTML.
`storeInCache`	boolean	true	Keep this response in the cache so a later request can be served free.
`timeout`	integer	90000	Give up on this request after this many milliseconds.
`waitFor`	integer	0	Milliseconds to wait after the page loads before reading it, for content that arrives late. Needs a real browser, so it is priced as a browser fetch.

Response

A success is always {"success": true, "data": {…}}. data carries what you asked for in formats, the page's metadata, and a cost object saying what the call was charged. A failure is {"success": false, "error": {…}} with a code you can branch on — see Errors.

Nothing is charged for a request that failed. A page you already fetched, re-read within maxAge, is free; one served from the shared index costs a single credit. The cost object says which it was.

Output formats

Crawl