Endpoints
Batch
Thousands of URLs as one job, for a backfill or a nightly refresh, running at whatever concurrency your plan allows. Poll it or take a webhook.
POST
/v1/batch/scrape
Request
Send a JSON body with a URL (or a list) and any options. The response holds data and what the call cost.
import requestsr = requests.post( 'https://api.snoopscan.com/v1/batch/scrape', headers={'Authorization': 'Bearer sk_YOUR_KEY'}, json={ 'urls': [ 'https://example.com/a', 'https://example.com/b' ], 'maxConcurrency': 10, 'scrapeOptions': { 'formats': [ 'markdown' ] } },)print(r.json())
const r = await fetch('https://api.snoopscan.com/v1/batch/scrape', { method: 'POST', headers: { Authorization: 'Bearer sk_YOUR_KEY', 'Content-Type': 'application/json' }, body: JSON.stringify({ "urls": [ "https://example.com/a", "https://example.com/b" ], "maxConcurrency": 10, "scrapeOptions": { "formats": [ "markdown" ] } }),});console.log(await r.json());
curl -X POST https://api.snoopscan.com/v1/batch/scrape \ -H 'Authorization: Bearer sk_YOUR_KEY' \ -H 'Content-Type: application/json' \ -d '{ "urls": [ "https://example.com/a", "https://example.com/b" ], "maxConcurrency": 10, "scrapeOptions": { "formats": [ "markdown" ] }}'
Returns job id · one job · from 1 credit a page.
Parameters
| Field | Type | Default | What it does |
|---|---|---|---|
urls
required |
array of string | — | The pages to scrape as one job, 1 to 10,000. |
maxConcurrency
|
integer | 10 | How many pages to fetch at once, 1 to 50. |
scrapeOptions
|
object | — | How each page is fetched and what comes back: the same options /v1/scrape takes. |
webhook
|
object | — | Where to send job events as pages complete. |
Response
A success returns {"success": true, "data": {…}}. data holds the formats you asked for, the page's metadata, and a cost object with what the call was charged. A failure returns {"success": false, "error": {…}} with an error code. See Errors.
Failed requests are not charged.
Re-reading a page you already fetched, within
maxAge, is free. See Credits for every rate.