Batch is available to every Zenrows account. You need an active Zenrows API key. The
ZenRowsBatchClient shown here is separate from the SDK’s synchronous scraping client, ZenRowsClient.Install and authenticate
Install the SDK from PyPI. The batch client ships in the samezenrows package as the synchronous client:
zenrows 1.10.0 or later. Webhook verification, BatchErrorCode, on_skip and the paused-run waiter behavior arrived in 1.10.0.
Python
with ZenRowsBatchClient(...) as client: ...) when you want connections to close automatically. It also takes timeout (seconds per API request, default 30) and retries (default 3). By default it targets the Batch API at https://async.api.zenrows.com/v1; pass base_url=... (or set ZENROWS_BATCH_BASE_URL) to point at a different environment.
Quickstart: submit, wait, get results
This is the standard end-to-end flow.submit_regular returns a JobRef (no GET yet), job.run.wait() blocks until the current run reaches a terminal state and returns a RunHandle, and run.results(...) iterates the per-task results.
Python
job.run.wait() returns a RunHandle whose .data snapshot is ready, so run.results(...) and run.download_to_dir(...) work without re-passing IDs. It returns on every terminal status (completed, stopped, failed), so check run.status. A run that ended failed stopped for an account reason: run.data.failure_reason is insufficient_credits, subscription_inactive or api_key_cap_reached, and run.data.failure_detail explains it.
A paused run makes no progress until someone resumes it, so the waiter doesn’t poll it until its timeout: it raises RunPaused (a WaiterError) with the paused run on .run. Pass wait_while_paused=True to keep polling, for example when another process resumes it.
Configuring the scrape
zenrows_params sets scraping options for every URL in the job. Values can be strings, booleans, or integers.
Python
response_type: markdown produces Markdown; extract, json_response, or a CSS extractor produce JSON; otherwise the result is HTML. Zenrows records this format on each result. Other common keys include mode, js_instructions, wait, block_resources, json_response, css_extractor, extract, extract_fields, outputs, original_status, allowed_status_codes, and session_id. The autoparse parameter is still accepted but deprecated. Batch doesn’t take the screenshot parameters, and the API rejects an unknown key with 400.
custom_headers accepts a dict of header name to value, for example {"custom_headers": {"Referer": "https://www.google.com/"}}, and is also how you set the Content-Type for a POST task. Up to 32 headers, 8 KB serialized.Per-URL options and correlating results
To set a per-URLexternal_id, metadata, or zenrows_params, pass dictionaries instead of plain strings. Per-URL zenrows_params override the job-level ones when a key appears in both. A task can also fetch its target with POST: set "method": "POST" and a "body" (a JSON value or a string, up to 16 KiB). A POST task can’t use js_render, js_instructions or json_response.
Python
Downloading content
results() gives you metadata plus a presigned result_url per task; the SDK downloads bodies straight from that URL with no extra API round-trip. Four options, on the current run (job.run.*) or any specific run (client.get_run(job_id, run_id=...)):
Python
result_url is a plain presigned URL (no auth header) valid for 2 hours, so fetch it promptly or re-list the results to get a fresh one.
Results are kept for 14 days by default. After that, a successful task reads expired (TaskStatus.EXPIRED) and has no result_url, and a status="successful" filter still returns it. download_to_dir and download_to_memory skip expired and failed rows, which have no body to fetch, pass each one to on_skip, and log one warning with the counts. Reading an expired task’s content through the API raises BatchAPIError with status 410 and code result_expired.
Open jobs: streaming URLs in batches
When you don’t have every URL up front, open a job, add tasks in batches, and close it. The run starts with the first tasks and completes once it finishes the final batch.Python
idempotency_key, a repeat of the same call adds its tasks once, so the SDK retries it on a transient failure. Without one, every call appends and a failed call isn’t retried. Setting "last_batch": True (or calling job.close()) locks the job, and the API rejects further adds. An open job counts as one of your active jobs until you delete it, or close it once its run has ended. File-input (CSV) upload isn’t supported for open jobs; use a closed job for CSV.
Large inputs: CSV upload
For lists too large to inline, upload a CSV once and reference it.upload_csv allocates the slot, PUTs your file to the presigned URL, and returns a file_input_id that submit_regular(file_input_id=...) consumes.
Python
url and optional external_id columns, and applies the job-level zenrows_params to all of them. Limits: 50 MB and 100,000 rows. The upload link works for 30 minutes and the uploaded file for 24 hours. A CSV can feed a closed regular job or a scheduled job.
Listing and pagination
The SDK offers both single-page calls (list_jobs, list_runs, list_results) and lazy iterators that page for you. The iterators yield loaded handles.
Python
Reruns and retrying failures
Each rerun creates a brand-new run on the same job. A job runs one run at a time: start a rerun only after the previous run reaches a terminal state, or the API answers409 run_not_terminal. Pick one of the two calls below.
Python
Python
202 and writes the new run’s tasks in the background, so results read straight after a rerun can be partial. wait_for_ingest=True returns only once they’re written. A partial rerun (retry_failed, or rerun(status=...)) must start within 10 days of when the run you retry started, or the API answers 409 partial_rerun_window_expired; a full rerun still works. When no task has the statuses you asked for, the API answers 409 no_matching_tasks.
Scheduled jobs
Build a schedule with the typed builders and submit a scheduled job. Zenrows stores its URLs as a template, and each fire produces a fresh run. ARate fires at least 60 minutes apart, Calendar times must be full hours, and an account holds up to 5 scheduled jobs.
Python
job.schedule.* controls the recurring schedule, and job.run.* controls the current run. This disambiguates the API’s two distinct “pause” actions:
Python
Completion notifications: webhooks
Attach a webhook at submit time, and Zenrows sends one event to your HTTPS URL when a run finishes on its own:run.completed when it completes, or run.failed when an account-level error ends it. Stopping a run yourself sends no event. A delivery that fails or takes longer than 10 seconds is retried with backoff for up to 3 days and then dropped, so the same event can arrive more than once: deduplicate on the X-ZenRows-Event-Id header. The body is a WebhookEvent (schema zenrows.webhook/v1) with the job and run ids, your external_id and metadata, the run’s stats, and, on run.failed, failure_reason and failure_detail.
Set signature: True to receive HMAC-signed deliveries (see Signed webhooks); it defaults to unsigned. Manage the webhook later with job.set_webhook(url, signature=...) / job.delete_webhook(), and probe a receiver with client.test_webhook(...).
Python
Bulk export (ZIP)
Bundle an entire run’s result bodies into a single server-side zip:download_all_results starts the export, waits for it, and streams the file to disk.
Python
The server-side zip is capped at 100 GiB per run. Its download link is valid for 2 hours, and the export for 12 hours. For larger runs, or one file per task, use
download_to_dir, which fetches bodies client-side with no size limit (slower; tune with concurrency). For manual control, client.start_results_export(...) and client.wait_for_export(...) expose the two steps separately.Estimating cost
Estimate credits before submitting withclient.estimate_cost. It takes the same body you would submit and runs client-side with no API call. The tiers are 1 (base), 5 (js_render), 10 (premium_proxy), 25 (both), and a dynamic 1 to 25 for mode=auto.
Python
min == max (est.exact) unless the job uses mode=auto, which yields a per-task range.
Checking actual cost
Once a run finishes, read the credits it consumed fromrun.stats.spend ({credits, cost}).
Python
404 or 410. Other failed tasks aren’t billed. Result rows don’t carry a spend value, so use the run-level rollup, or a task’s attempt history (run.task_history(task_id)) for its per-attempt spend. stats.spend is indicative; your Zenrows account statement is the authoritative record for billing.
Signed webhooks: HMAC keys
Signed deliveries (see Webhooks) need an active HMAC key for your organization. Rotation uses two slots, active plus candidate, so you can roll keys without downtime.Python
X-Signature: t=<unix>,v1=<hex>,kid=<key id>. verify_webhook_signature(raw_body, header, secrets) checks it: it picks the secret by kid, decodes it from base64 (the secret is base64-encoded), recomputes HMAC-SHA256(key, f"{t}.{raw_body}"), compares in constant time, and rejects a t more than 300 seconds from now (tolerance_seconds changes that). It returns False instead of raising. Pass the raw body, before any JSON parsing, and keep both keys in secrets during a rotation.
Python
Error handling
Every non-2xx raisesBatchAPIError. Its code attribute carries the stable code from the problem body, status_code the HTTP status, and detail the explanation. BatchErrorCode names every code the API sends; its members are strings, so they compare equal to code. The error code reference explains each one.
Python
quota_exceeded is one of three account limits: 3 active jobs (shared with Crawl), 5 scheduled jobs, or 100,000 tasks in one open job. It clears only when a job’s run ends (for an open job, once it’s also closed) or a job is deleted, so the client doesn’t retry it. Other transient failures (network errors, 502, 503, 504, and any other 429) are retried with backoff on reads and on the calls that carry an idempotency_key.
Task-level failures don’t raise exceptions. They appear as failed results with an error object you can read through task.error.code and task.error.detail.
Idempotency
Passidempotency_key to submit_*, to rerun / retry_failed, and to add_tasks to make a retry safe. The API keeps each key for 24 hours:
- The same key with the same request gets the original response replayed, so a dropped connection never creates a second job, run or set of tasks.
- The same key with a different request, or a repeat while the first call is still being processed, raises a
409with codeidempotency_key_conflict. - A submit or rerun the API refused, for example with
429 quota_exceeded, doesn’t use up its key: fix the cause and retry with the same key. An add-tasks call keeps its key, refused or not: a retry with the same body is checked again, and a corrected body needs a new key. - A key can be up to 128 characters.
Python
ZenRowsBatchClient. See the SDK repository or run help(ZenRowsBatchClient).