Introduction to Batch

Zenrows Batch is a managed service that scrapes large lists of URLs for you. You hand it the URLs and the scraping configuration you want, and the API runs the work reliably: it queues each request, retries transient failures, stores every result, and notifies you when the batch finishes. Whether you scrape a few hundred pages or a few hundred thousand, Batch gives you one consistent way to submit work, track progress, and collect results, without operating the orchestration yourself.

​

Use It from Any Language

Every Batch feature is available over a plain HTTPS REST API, so you can integrate from any language with nothing more than an HTTP client. The API reference shows every request in Python, Node.js, Java, PHP, Go, Ruby and cURL. Official Python, Node.js and Go SDKs wrap the same calls. The CLI and the MCP server cover submitting, tracking and recovering jobs; schedules, webhooks, exports, CSV inputs and open jobs are API and SDK only.

​

Why Use It

Scraping a single page is easy. Scraping tens of thousands of pages reliably is not: you have to spread the load so you don’t overwhelm target sites, retry the requests that fail for transient reasons, store and serve results durably, and know when everything finished and what didn’t work. Batch is the managed layer for exactly this. You integrate once, submit jobs, and it handles the operational concerns for you:

• Scale without orchestration. Submit up to 100,000 URLs in a single job and let the API pace the work.
• Reliable by default. The API retries transient failures automatically, so a page that’s briefly unavailable still gets fetched.
• Results you can trust. The API stores every successful page and serves it back through time-limited download links.
• Know when it’s done. Receive a webhook when a run finishes, or poll for status, whichever fits your pipeline.

​

How It Works

1. Submit a job: a list of URLs plus the scraping options to apply to them.
2. The API queues and processes each URL, calling the Zenrows scraping engine and retrying transient failures automatically.
3. The API stores results as they complete, so you can watch progress at any time.
4. Collect the results when the batch finishes. Fetch them directly, or receive a webhook. Results stay available for 14 days.

​

Core Concepts

Three terms map cleanly onto how teams work: A job carries a snapshot of its latest run, so a single request tells you where things stand without extra lookups.

​

Key Features

​

Flexible submission

Submit up to 100,000 URLs in a single job. An open job accepts URLs in batches over time (up to 10,000 per add-tasks call and 100,000 in total) before you close it. For very large lists, upload a CSV (up to 50 MB / 100,000 rows) and reference it instead of inlining URLs. Send an optional idempotency key with a submit, rerun or add-tasks call so a retry is safe: the same key and request within 24 hours returns the original response instead of doing the work twice, and the same key with a different request returns 409. A task can also fetch its target with POST, with a body of up to 16 KiB.

​

Full control over scraping

Configure each job, and override per URL, with the scraping options you need:

• JavaScript rendering for dynamic pages.
• Premium proxies and geo-targeting by country.
• Output formats: raw HTML, parsed JSON, Markdown, plain text, or PDF.
• Structured extraction with CSS selectors or automatic parsing, plus custom headers, waits, and more.

​

Results, your way

• Each successful task yields a presigned download URL (a temporary, signed link) valid for 2 hours, or you can stream the content directly.
• Page through results efficiently and filter by status: successful, failed, or in-progress.
• Correlate every result back to your own records with a caller-supplied external_id.
• Archive export: download an entire run’s results as a single ZIP archive in one request.
• Each task ends as successful or failed. Results are kept for 14 days by default after each task finishes; after that, a successful task’s row reads expired, has no download link, and its content returns 410 result_expired. A failed task carries a structured error with a code and detail. These codes don’t yet reliably distinguish blocked, not-found, and timeout, so treat them as a hint rather than a definitive reason.

​

Completion notifications

• Webhooks: the API sends an HTTPS callback when a run completes (run.completed) or an account-level error ends it (run.failed), optionally HMAC-signed so your receiver can verify it genuinely came from Zenrows. Failed deliveries are retried for up to 3 days.
• Polling: status and progress counters are always available on demand.

​

Reruns and run control

• Full rerun: replay an entire job as a fresh run.
• Retry failed only: re-execute just the tasks that failed (and optionally those that never started), within 10 days of when the run you retry started. Successful results carry over, so you only pay to re-scrape what failed.
• Pause, resume, or stop a run: reversibly suspend an expensive batch and resume it later, or stop it entirely, without waiting for it to finish.

​

Scheduled jobs

Run recurring work on a one-shot, interval (every N minutes, hours, or days, minimum 60 minutes between runs), or calendar (specific times on specific days) schedule. Schedules are timezone-aware using IANA names, so daylight-saving transitions stay correct. Pause and resume any time without losing the schedule.

​

Cost transparency

Batch uses the same rate card as Fetch, and it’s small and stable, so you can estimate the cost before you submit with no extra round-trip and show your users an accurate credit range up front. Once a job finishes, you can also see the total credits it actually consumed, which makes it easy to reconcile the estimate against realized spend.

​

Reliability

• Pages still come back. The API retries transient failures for you, so a page that’s temporarily unavailable still has a strong chance of being fetched.
• Easy on your targets. The API paces requests so a large job stays polite to the sites you scrape.
• Durable results. The API stores every successful page and serves it back through secure, time-limited download links.
• Built for big batches. Submit tens of thousands of URLs and collect the results without managing any of the moving parts.

​

Security

• API-key authentication on every request. Keys travel in a header, never in the URL.
• HTTPS everywhere, including a requirement that webhook receivers use HTTPS.
• Optional HMAC-signed webhooks with per-organization keys and phased key rotation, so you can verify that a callback genuinely came from Zenrows.

​

Limits

An account runs 3 active jobs by default, shared with Crawl. One more returns 429 quota_exceeded, as does a sixth scheduled job or a task past an open job’s 100,000. The detail says which limit. An open job counts as active even after you stop its run. A job holds a slot until you delete it, or until it’s closed and its latest run has ended (completed, stopped or failed); deleting frees it at once. An open job that receives no tasks for 48 hours closes automatically. To run more than 3 at once, use the chat in your dashboard. Email success@zenrows.com if you can’t use the chat.

​

Pricing

Pricing is identical to Zenrows Fetch, in credits. Successful tasks are billed, and so are tasks that fail because the target answered 404 Not Found or 410 Gone, as on Fetch; Batch records those tasks as failed because they have no usable content. Other failed tasks aren’t billed. The credit cost depends on the scraping tier each URL uses: Optimize cost with Adaptive Stealth Mode. Batch supports Adaptive Stealth Mode (mode=auto): it starts each request with the cheapest viable setup and escalates to JavaScript rendering or premium proxies only when a site actually needs them. You pay for the lightest configuration that succeeds rather than over-provisioning every URL. Because the rate card is small and stable, you can present an accurate cost range to your own users before a job is submitted. See Estimate cost.

​

OpenAPI Spec

The Batch API is described in OpenAPI 3.1 at www.zenrows.com/openapi.json, alongside Fetch. Batch operations declare their own server, https://async.api.zenrows.com/v1, so a generated client sends them to the right host. The spec is listed in the RFC 9727 API catalog at www.zenrows.com/.well-known/api-catalog.

​

Next Steps