Python SDK

The Batch Scraper API is in public beta, available to every ZenRows account. You need an active ZenRows API key. The ZenRowsBatchClient shown here is separate from the SDK’s synchronous scraping client, ZenRowsClient.

Use this guide to submit batch scraping jobs, track them, and collect results with the official ZenRows Python SDK. It walks through every Batch Scraper API feature, and each section runs on its own.

Prefer raw HTTP, or work in another language? The REST API guide covers the same use cases against the plain REST API. For the complete method reference, see the SDK repository on GitHub.

Install and Authenticate

While the batch client is being published to PyPI, install the SDK from the beta branch:

pip install "git+https://github.com/ZenRows/zenrows-python-sdk.git@feature/batch-api/rc"

This git install is temporary. The batch client is not on PyPI yet; once it is published, pip install zenrows will include it.

Python

import os
from zenrows import ZenRowsBatchClient

client = ZenRowsBatchClient(api_key=os.environ["ZENROWS_API_KEY"])

The client also works as a context manager (with ZenRowsBatchClient(...) as client: ...) when you want connections to close automatically. By default it targets the production Batch Scraper API; pass base_url=... (or set ZENROWS_BATCH_BASE_URL) to point at a different environment.

Quickstart: Submit, Wait, Get Results

This is the standard end-to-end flow. submit_regular returns a JobRef (no GET yet), job.run.wait() blocks until the current run reaches a terminal state and returns a RunHandle, and run.results(...) iterates the per-task results.

Python

job = client.submit_regular(
    [
        "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie",
        "https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket",
        "https://www.scrapingcourse.com/ecommerce/product/aero-daily-fitness-tee",
    ],
    zenrows_params={"js_render": "true", "premium_proxy": "true"},
)

run = job.run.wait(timeout=600.0)   # blocks until the current run is terminal
s = run.stats
print(f"{s.successful}/{s.total} successful, {s.failed} failed")

for task in run.results(status="successful"):
    print(task.external_id or task.task_id, "->", task.result_url)

job.run.wait() returns a RunHandle whose .data snapshot is ready, so run.results(...) and run.download_to_dir(...) work without re-passing IDs.

Pass progress=True to the waiter for a live progress bar on long batches. For very large submissions that return 202 Accepted, pass wait_for_ingest=True to submit_regular so results pages are complete before you read them.

Configuring the Scrape

zenrows_params sets scraping options for every URL in the job. Values can be strings, booleans, or integers; the SDK converts them to strings before sending.

Python

job = client.submit_regular(
    ["https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie"],
    zenrows_params={
        "js_render": "true",          # render JavaScript in a headless browser
        "premium_proxy": "true",      # route through premium residential proxies
        "proxy_country": "us",        # geo-target the request
        "wait_for": ".price",         # wait for this CSS selector before capturing
        "response_type": "markdown",  # html | markdown | plaintext | pdf
    },
)

ZenRows derives the job’s output format from these options: response_type: markdown produces Markdown; autoparse, json_response, or a CSS extractor produce JSON; otherwise the result is HTML. ZenRows records this format on each result. Other common keys include js_instructions, wait, block_resources, json_response, css_extractor, autoparse, outputs, original_status, allowed_status_codes, and session_id. Custom request headers (custom_headers) are not supported yet.

Per-URL Options and Correlating Results

To set a per-URL external_id, metadata, or zenrows_params, pass dictionaries instead of plain strings. Per-URL zenrows_params override the job-level ones when a key appears in both.

Python

job = client.submit_regular(
    [
        {"url": "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short",
         "external_id": "order-42",
         "zenrows_params": {"proxy_country": "de"}},
        {"url": "https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee",
         "external_id": "order-43",
         "metadata": {"sku": "ABC-1"}},
    ],
    zenrows_params={"js_render": "true"},   # applies to both unless overridden
)

ZenRows returns your external_id unchanged on every result, so you can match results back to your own records without tracking the API’s task IDs. It must match ^[A-Za-z0-9._-]+$ and be 128 characters or fewer.

Downloading Content

results() gives you metadata plus a presigned result_url per task; the SDK downloads bodies straight from that URL with no extra API round-trip. Four options, on the current run (job.run.*) or any specific run (client.get_run(job_id, run_id=...)):

You want	Use	Where it lands
Every body, one file per task	`run.download_to_dir(dir)`	files on disk
Every body, kept in memory	`run.download_to_memory()`	`list[DownloadedResult]`
The whole run as one artifact	`run.download_all_results("out.zip")`	one `.zip`
One task’s body (inside a loop)	`run.download_task_to_file(task, path)` / `run.download_task_to_memory(task)`	file / `bytes`

Python

# Bulk to disk, named by external_id, 8 downloads in parallel:
n = run.download_to_dir(
    "./out",
    status="successful",       # default; pass status=None for everything
    use_external_id=True,      # name files <external_id>.<ext> instead of <task_id>.<ext>
    concurrency=8,
    progress=True,
    max_files=20_000,          # safety caps that raise DownloadLimitExceeded
    max_bytes_per_file=10 * 1024 * 1024,
)
print(f"downloaded {n} files")

# One task at a time, when you want a Python-side filter:
for task in run.results(status="successful"):
    body = run.download_task_to_memory(task)   # bytes; or download_task_to_file(task, path)

Each result_url is a plain presigned URL (no auth header) valid for a short window, so fetch it promptly or re-list the results to get a fresh one.

Open Jobs: Streaming URLs in Batches

When you don’t have every URL up front, open a job, add tasks in batches, and close it. The run stays active and completes once it finishes the final batch.

Python

job = client.submit_open(zenrows_params={"js_render": "true"})

job.add_tasks({"tasks": [{"url": "https://www.scrapingcourse.com/ecommerce/page/1"}]})
job.add_tasks({"tasks": [{"url": "https://www.scrapingcourse.com/ecommerce/page/2"}],
               "last_batch": True})  # closes the job

run = job.run.wait(timeout=900.0)

Setting last_batch=True (or calling job.close()) locks the job, and the API rejects further adds. File-input (CSV) upload is not supported for open jobs; use a closed job for CSV.

Large Inputs: CSV Upload

For lists too large to inline, upload a CSV once and reference it. upload_csv allocates the slot, PUTs your file to the presigned URL, and returns a file_input_id that submit_regular(file_input_id=...) consumes.

Python

file_input_id = client.upload_csv(
    "urls.csv",
    fields={"url": "URL", "external_id": "Customer Ref"},  # column name (header=True) or 0-based index
    header=True,
)
job = client.submit_regular(file_input_id=file_input_id, zenrows_params={"js_render": "true"})

Each CSV row becomes a task. ZenRows reads only the url and optional external_id columns, and applies the job-level zenrows_params to all of them. Limits: 50 MB / 100,000 rows.

Listing and Pagination

The SDK offers both single-page calls (list_jobs, list_runs, list_results) and lazy iterators that page for you. The iterators yield loaded handles.

Python

for job in client.iter_jobs(job_type="regular", status="closed"):
    print(job.job_id, job.data.status)

for run in client.iter_runs("01J...", page_size=50):
    print(run.run_id, run.data.status)

for task in client.iter_results("01J...", status="failed"):
    print(task.task_id, task.error.code if task.error else None)

Reruns and Retrying Failures

Each rerun creates a brand-new run on the same job.

Python

# Re-run every task:
new_run = job.rerun()

# Re-run only the failures (optionally also tasks that never started).
# Successful results carry over, so you don't pay to scrape them again:
new_run = job.retry_failed(include_pending=False)
new_run.wait(timeout=600.0)

retry_failed() is the usual post-run cleanup, a thin shortcut for rerun(status="failed"). You can act on an id you kept without a preceding GET: client.job(job_id).retry_failed().

Scheduled Jobs

Build a schedule with the typed builders and submit a scheduled job. ZenRows stores its URLs as a template, and each fire produces a fresh run.

Python

from datetime import datetime
from zenrows.batch import At, Calendar, Rate, Weekly

# Every 6 hours:
client.submit_scheduled(Rate(every=6, unit="hour"),
                        ["https://www.scrapingcourse.com/ecommerce/"])

# 09:00 and 18:00 on Mon/Wed/Fri, Berlin time:
client.submit_scheduled(
    Calendar(times_of_day=["09:00", "18:00"],
             cadence=Weekly(days=["mon", "wed", "fri"]),
             timezone="Europe/Berlin"),
    ["https://www.scrapingcourse.com/ecommerce/"],
)

# One-shot at a wall-clock time. A naive datetime plus an IANA zone
# keeps daylight-saving transitions correct:
client.submit_scheduled(At(datetime(2026, 9, 1, 9, 0), timezone="Europe/Berlin"),
                        ["https://www.scrapingcourse.com/ecommerce/"])

A job carries two facets, matching the API’s two scopes. job.schedule.* controls the recurring schedule, and job.run.* controls the current run. This disambiguates the API’s two distinct “pause” actions:

Python

job.schedule.pause()                            # stop firing new runs
job.schedule.resume()
job.schedule.update(Rate(every=1, unit="day"))  # replace the schedule

job.run.pause()   # reversibly suspend the current run (a different endpoint)
job.run.resume()
job.run.stop()    # end the current run

Completion Notifications: Webhooks

Attach a webhook at submit time, and ZenRows sends a run.completed event to your HTTPS URL when a run finishes. Set signature: True to receive HMAC-signed deliveries (see Signed Webhooks); it defaults to unsigned. You can also manage the webhook later with client.put_job_webhook(...) / client.delete_job_webhook(...), and probe a receiver with client.test_webhook(...).

Python

job = client.submit_regular(
    ["https://www.scrapingcourse.com/ecommerce/"],
    webhook={"url": "https://your-app.com/webhooks/zenrows", "signature": True},
)

ZenRows delivers each event at least once, so your receiver must deduplicate on the X-ZenRows-Event-Id header.

No webhook receiver? Just poll for completion with job.run.wait() or run.wait().

Bulk Export (ZIP)

Bundle an entire run’s result bodies into a single server-side zip: download_all_results starts the export, waits for it, and streams the file to disk.

Python

run.download_all_results("results.zip")

The server-side zip is capped at 1 GiB per run. For larger runs, or one file per task, use download_to_dir, which fetches bodies client-side with no size limit (slower; tune with concurrency). For manual control, client.start_results_export(...) and client.wait_for_export(...) expose the two steps separately.

Estimating Cost

Estimate usage before submitting with client.estimate_cost. It takes the same body you would submit and runs client-side with no API call. The tiers are 1 (base), 5 (js_render), 10 (premium_proxy), 25 (both), and a dynamic 1 to 25 for mode=auto.

Python

est = client.estimate_cost({
    "type": "regular",
    "tasks": [{"url": f"https://www.scrapingcourse.com/ecommerce/page/{i}"} for i in range(1000)],
    "zenrows_params": {"js_render": "true"},
})
print(est)                       # "5000 credits (1000 tasks)"
print(est.min, est.max, est.exact)
print(est.format())              # per-tier breakdown table

An estimate assumes each URL succeeds once. min == max (est.exact) unless the job uses mode=auto, which yields a per-task range.

Checking Actual Cost

Once a run finishes, read the actual usage it consumed from run.stats.spend ({credits, cost}).

Python

run = job.run.wait(timeout=600.0)
spend = run.stats.spend
if spend:
    print(f"this run used {spend.credits} usage (cost {spend.cost})")

Only successful requests are charged, so realized spend is at most the estimate. Per-task task.spend is indicative and often absent on successful rows, so rely on the run-level rollup for actuals. Your ZenRows account statement remains the authoritative record for billing.

Signed Webhooks: HMAC Keys

Signed deliveries (see Webhooks) need an active HMAC key for your organization. Rotation uses two slots, active plus candidate, so you can roll keys without downtime.

Python

created = client.rotate_hmac_key()   # the first call creates the active key
print(created.kid, created.secret)   # ZenRows returns the secret ONCE, so store it now

# Phased rotation:
client.rotate_hmac_key()             # stage a candidate (start accepting both)
client.finalize_hmac_key()           # promote the candidate to active, retire the old one
# client.cancel_hmac_rotation()      # or discard the candidate
client.list_hmac_keys()              # metadata only; never returns secrets

Your receiver verifies the X-Signature: t=<unix>,v1=<hex>,kid=<active> header by recomputing HMAC-SHA256(secret, f"{t}.{raw_body}") and comparing the result to v1, selecting the secret by kid.

Error Handling

Every non-2xx raises BatchAPIError. Its code attribute carries the stable code from the RFC 7807 body, and status_code the HTTP status.

Python

from zenrows.batch import BatchAPIError

try:
    client.get_job("does-not-exist")
except BatchAPIError as e:
    print(e.status_code, e.code)   # e.g. 404 not_found
    raise

Task-level failures don’t raise exceptions. They appear as failed results with an error object you can read through task.error.code and task.error.detail.

Idempotency

Pass idempotency_key to submit_* and to rerun / retry_failed to guard against accidentally creating duplicate jobs. Each key can be used once: a later call that reuses the same key raises a 409 conflict, whether or not the body is identical, and the API does not store or replay the original result. An idempotency key is therefore not a safe-retry token, so if a call fails midway, check whether the job was created before retrying with a new key.

Python

job = client.submit_regular(
    ["https://www.scrapingcourse.com/ecommerce/"],
    idempotency_key="nightly-2026-06-08",
)

The full Batch surface (jobs, runs, tasks, results, content, history, file inputs, and HMAC keys) is reachable through methods on ZenRowsBatchClient. See the SDK repository or run help(ZenRowsBatchClient).

First Steps

Universal Scraper API

Scraping Browser

Residential Proxies

Batch Scraper API (Beta)

Account

Install and Authenticate

Quickstart: Submit, Wait, Get Results

Configuring the Scrape

Per-URL Options and Correlating Results

Downloading Content

Open Jobs: Streaming URLs in Batches

Large Inputs: CSV Upload

Reruns and Retrying Failures

Scheduled Jobs

Completion Notifications: Webhooks

Bulk Export (ZIP)

Estimating Cost

Checking Actual Cost

Signed Webhooks: HMAC Keys

Error Handling

Idempotency

​Install and Authenticate

​Quickstart: Submit, Wait, Get Results

​Configuring the Scrape

​Per-URL Options and Correlating Results

​Downloading Content

​Open Jobs: Streaming URLs in Batches

​Large Inputs: CSV Upload

​Listing and Pagination

​Reruns and Retrying Failures

​Scheduled Jobs

​Completion Notifications: Webhooks

​Bulk Export (ZIP)

​Estimating Cost

​Checking Actual Cost

​Signed Webhooks: HMAC Keys

​Error Handling

​Idempotency

Install and Authenticate

Quickstart: Submit, Wait, Get Results

Configuring the Scrape

Per-URL Options and Correlating Results

Downloading Content

Open Jobs: Streaming URLs in Batches

Large Inputs: CSV Upload

Listing and Pagination

Reruns and Retrying Failures

Scheduled Jobs

Completion Notifications: Webhooks

Bulk Export (ZIP)

Estimating Cost

Checking Actual Cost

Signed Webhooks: HMAC Keys

Error Handling

Idempotency