Skip to main content
Batch is available to every Zenrows account. You need an active Zenrows API key. The ZenRowsBatchClient shown here is exported from the main zenrows package, next to the SDK’s synchronous scraping client, ZenRows. Node.js 20 or later is required.
Use this guide to submit batch scraping jobs, track them, and collect results with the official Zenrows Node.js SDK. The client is fully typed. Handles expose camelCase properties (job.jobId, run.runId), while the data they carry keeps the API’s snake_case field names (run.data.stats.successful, row.result_url, row.external_id).
Prefer raw HTTP, or work in another language? The REST API guide covers the same use cases against the plain REST API. Limits are in the introduction and error codes in API error codes. For the complete method reference, see the SDK repository on GitHub.

Install and authenticate

This page describes zenrows 2.4.1 or later. verifyWebhookSignature, BatchErrorCode, getTaskContentBytes, onSkip and the paused-run waiter behavior arrived in 2.4.1.
If you already use the synchronous client, the same batch client is available as new ZenRows("YOUR_ZENROWS_API_KEY").batch. The constructor takes { baseURL, retries, timeout } as its second argument. retries defaults to 3 and timeout to 30 seconds per API request. The timeout doesn’t limit how long a waiter runs or how long a result download takes.

Quickstart: submit, wait, get results

submitRegular(urls, fileInputId, options) takes the task list first (bare URL strings or per-task objects), an optional CSV fileInputId second, and job options third. It returns a JobRef. job.run.wait() blocks until the current run is terminal and returns its RunHandle, whose .data snapshot is ready. results() is an async generator you iterate with for await.
wait() polls for up to 300 seconds by default and then throws WaiterTimeoutError. Pass timeout in seconds for longer jobs. It resolves on every terminal status (completed, stopped, failed), so check run.status. A run that ended failed stopped for an account reason: run.data.failure_reason is insufficient_credits, subscription_inactive or api_key_cap_reached, and run.data.failure_detail explains it. A paused run makes no progress until someone resumes it, so wait() doesn’t poll it until the timeout: it throws a RunPausedError whose run is the paused run. Pass waitWhilePaused: true to keep polling, for example when another process resumes it.
Need full control over the request body? client.submitJob({...}) accepts the raw wire shape (type, status, tasks, zenrows_params, and so on) and returns the same JobRef. A submit of 10,000 tasks or more returns 202 Accepted while the API is still writing the run’s tasks (ingest_status: "pending"). Pass waitForIngest: true so results pages are complete before you read them.

Configuring the scrape

zenrowsParams sets scraping options for every URL in the job. The parameter keys themselves stay in their API form (js_render, premium_proxy, and so on).
Other common keys include mode, js_instructions, wait, block_resources, json_response, css_extractor, extract, extract_fields, outputs, original_status, allowed_status_codes, and session_id. The autoparse parameter is still accepted but deprecated. Batch doesn’t take the screenshot parameters, and the API rejects an unknown key with 400. Zenrows derives the output format from these and records it on each result.
custom_headers takes a map of header name to value, for example { custom_headers: { Referer: "https://www.google.com/" } }. It is also how you set the Content-Type for a POST task. Up to 32 headers, 8 KB serialized.

Per-URL options and correlating results

To set a per-task external_id, metadata, or zenrows_params, pass objects instead of bare strings. Per-task zenrows_params override the job-level zenrowsParams on a key clash. A task can also fetch its target with POST: set method: "POST" and a body (a JSON value or a string, up to 16 KiB). A POST task can’t use js_render, js_instructions or json_response.
The API returns your external_id unchanged on every result, so you can match results back to your own records. It must match ^[A-Za-z0-9._-]+$ and be 128 characters or fewer, and it doesn’t have to be unique. metadata takes up to 20 string values.

Downloading content

results() gives you metadata plus a presigned result_url per task, valid for 2 hours. The SDK downloads bodies straight from that URL with no extra API round-trip. Run these on the current run (job.run.*) or any specific run (client.run(jobId, runId)).
Results are kept for 14 days by default. After that, a successful task reads expired and has no result_url, and a status: "successful" filter still returns it. downloadToDir and downloadToMemory skip expired and failed rows, which have no body to fetch, and pass each one to onSkip. To read one task’s content through the API instead, client.getTaskContent(jobId, taskId) returns it as text. For a pdf or other binary result, use client.getTaskContentBytes(jobId, taskId), which returns a Buffer. Both throw a ZenRowsBatchError: 409 task_not_ready while the task runs, 422 with the task’s error for a failed task, and 410 result_expired past the retention window.
downloadAllResults is a server-side zip capped at 100 GiB per run. For larger runs, or one file per task, use downloadToDir, which fetches bodies client-side with no size limit.

Open jobs: streaming URLs in batches

When you don’t have every URL up front, open a job with submitOpen, add tasks in batches, and close it. The run starts with the first tasks and completes once it finishes the final batch.
Each call takes up to 10,000 URLs, and an open job holds up to 100,000 in total. With an idempotencyKey, a repeat of the same call adds its tasks once, so the SDK retries it on a transient failure. Without one, every call appends and a failed call isn’t retried. Setting lastBatch: true (or calling job.close()) locks the job, and the API rejects further adds. An open job counts as one of your active jobs until you delete it, or close it once its run has ended. CSV file input isn’t supported for open jobs; use a closed job for CSV.

Large inputs: CSV upload

For lists too large to inline, upload a CSV once and reference it. uploadCsv takes the file’s contents, allocates the slot, PUTs the contents to the presigned URL, and returns a fileInputId you pass to a submit call.
Each CSV row becomes a task. Only the url and optional external_id columns are read, and the job-level zenrowsParams apply to all of them. header defaults to false, so set it when the first row holds column names. Limits: 50 MB and 100,000 rows. The upload link works for 30 minutes and the uploaded file for 24 hours. A CSV can feed a closed regular job or a scheduled job.

Listing and pagination

The SDK offers both single-page calls (listJobs, listRuns, getResults) and auto-paginating async generators that yield loaded handles or result rows.

Reruns and retrying failures

Each rerun creates a brand-new run on the same job. A job runs one run at a time: start a rerun only after the previous run reaches a terminal state, or the API answers 409 run_not_terminal. Pick one of the two calls below.
The API accepts every rerun with 202 and writes the new run’s tasks in the background, so results() read straight after a rerun can be partial. waitForIngest: true resolves only once they’re written. A partial rerun (retryFailed, or rerun({ status })) must start within 10 days of when the run you retry started, or the API answers 409 partial_rerun_window_expired; a full rerun still works. When no task has the statuses you asked for, the API answers 409 no_matching_tasks.

Scheduled jobs

Submit a recurring job with a typed schedule. Build the schedule with one of three classes: Rate (every N minutes, hours, or days, at least 60 minutes apart), At (one-shot, at a wall-clock time in a timezone), or Calendar (full-hour times of day on a Daily, Weekly, or Monthly cadence, in a timezone).
A job carries two facets that match the API’s two scopes. job.schedule.* controls the recurring schedule, and job.run.* controls the current run. This disambiguates the API’s two distinct “pause” actions:
An account holds up to 5 scheduled jobs. For a specific historical run, use client.run(jobId, runId), a RunRef with read and download operations only.

Completion notifications: webhooks

Attach a webhook at submit time, and Zenrows POSTs one event to your HTTPS URL when a run finishes on its own: run.completed when it completes, or run.failed when an account-level error ends it. Stopping a run yourself sends no event. A delivery that fails or takes longer than 10 seconds is retried with backoff for up to 3 days and then dropped, so the same event can arrive more than once: deduplicate on the X-ZenRows-Event-Id header. The body is a WebhookEvent (schema zenrows.webhook/v1) with the job and run ids, your external_id and metadata, the run’s stats, and, on run.failed, failure_reason and failure_detail.
Set signature: true for HMAC-signed deliveries (see Signed webhooks); it defaults to unsigned. Manage the webhook later with job.setWebhook(url, signature), job.getWebhook() and job.deleteWebhook(), and probe a receiver with client.testWebhook(...). client.putJobWebhook(jobId, { url, signature }) also works; if you leave signature out, the SDK reads the job’s current setting and sends it back, so changing only the URL keeps signing as it was.
No webhook receiver? Just poll for completion with job.run.wait() or run.wait().

Bulk export (ZIP)

Bundle an entire run’s result bodies into a single server-side zip: downloadAllResults starts the export, waits for it, and streams the file to disk.
The server-side zip is capped at 100 GiB per run. Its download link is valid for 2 hours, and the export for 12 hours. For larger runs, or one file per task, use downloadToDir, which fetches bodies client-side with no size limit. For manual control, client.startResultsExport(...) and client.waitForExport(...) expose the two steps separately.

Estimating cost

Estimate credits before submitting with client.estimateCost(tasks, zenrowsParams). It takes the same task list as submitRegular plus the job-level params, and runs client-side with no API call. The tiers are 1 (base), 5 (js_render), 10 (premium_proxy), 25 (both), and a dynamic 1 to 25 for mode=auto.
An estimate assumes each URL succeeds once. est.exact is true and min equals max unless the job uses mode=auto, which yields a range. A flag counts as on only when it is true or "true", the values the API accepts.

Checking actual cost

Once a run finishes, read the credits it consumed from run.data.stats.spend ({ credits, cost }).
Batch bills like Fetch. Successful tasks are billed, and so are tasks that fail because the target answered 404 or 410. Other failed tasks aren’t billed. Result rows don’t carry a spend field, so use the run-level rollup, or a task’s attempt history (client.getTaskHistory(jobId, taskId)) for its per-attempt spend. stats.spend is indicative; your Zenrows account statement is the authoritative record for billing.

Signed webhooks: HMAC keys

Signed deliveries (see Webhooks) need an active HMAC key for your organization. Rotation uses two slots, active plus candidate, so you can roll keys without downtime.
Each signed delivery carries X-Signature: t=<unix>,v1=<hex>,kid=<key id>. verifyWebhookSignature(rawBody, header, secrets) checks it: it picks the secret by kid, decodes it from base64, recomputes HMAC-SHA256(key, "<t>.<raw_body>"), compares in constant time, and rejects a t more than 300 seconds from now (toleranceSeconds changes that). It returns false instead of throwing. Pass the raw body, before any JSON parsing, and keep both keys in secrets during a rotation.

Error handling

Every non-2xx surfaces as ZenRowsBatchError. Its code property carries the stable code from the problem body, status the HTTP status, detail the explanation, retryAfter the Retry-After seconds when sent, and problem the full body. BatchErrorCode names every code the API sends, so you can branch without bare strings. The error code reference explains each one.
quota_exceeded is one of three account limits: 3 active jobs (shared with Crawl), 5 scheduled jobs, or 100,000 tasks in one open job. It clears only when a job’s run ends (for an open job, once it’s also closed) or a job is deleted, so the client doesn’t retry it. Task-level failures don’t throw. They appear as failed rows in results(), each with an error object you can read through row.error?.code and row.error?.detail.

Timeouts and retries

Transient failures (network errors, 502, 503, 504, and any 429 other than quota_exceeded) are retried automatically with jittered exponential backoff, honoring Retry-After. Retries apply only to idempotent requests: GET / PUT / DELETE, plus the POSTs that carry an idempotencyKey (submit, rerun and addTasks).
Waiters take their own limit in seconds: wait() and waitForIngest() take timeout, and downloadAllResults() takes waitTimeout (default 600).

Idempotency

Pass idempotencyKey to a submit, a rerun or addTasks to make a retry safe. The API keeps each key for 24 hours:
  • The same key with the same request gets the original response replayed, so a dropped connection never creates a second job, run or set of tasks.
  • The same key with a different request, or a repeat while the first call is still being processed, returns 409 idempotency_key_conflict.
  • A submit or rerun the API refused, for example with 429 quota_exceeded, doesn’t use up its key: fix the cause and retry with the same key. An add-tasks call keeps its key, refused or not: a retry with the same body is checked again, and a corrected body needs a new key.
  • A key can be up to 128 characters.
The full Batch surface (jobs, runs, tasks, results, content, history, file inputs, and HMAC keys) is reachable through methods on ZenRowsBatchClient. See the SDK repository on GitHub.