The Batch Scraper API is in private beta. You need an invited account and an active ZenRows API key. Replace
YOUR_ZENROWS_API_KEY in every example with your key.Basics
- Base URL:
https://async.api.zenrows.com/v1 - Authentication: send
X-API-Key: YOUR_ZENROWS_API_KEYon every request. The key travels in a header, never in the URL. - Bodies: send JSON with
Content-Type: application/jsonand receive JSON. Errors come back asapplication/problem+json, a standard JSON error format (RFC 7807) with a stablecodeyou can branch on.
Content-Type header when there is a body, and turns error responses into thrown objects:
Node.js
const BASE = process.env.BATCH_API_URL ?? "https://async.api.zenrows.com/v1";
const API_KEY = process.env.ZENROWS_API_KEY;
async function api(method, path, { body, query, headers } = {}) {
const url = new URL(BASE + path);
for (const [k, v] of Object.entries(query ?? {})) {
if (v !== undefined && v !== null) url.searchParams.set(k, v);
}
const res = await fetch(url, {
method,
headers: {
"X-API-Key": API_KEY,
...(body ? { "Content-Type": "application/json" } : {}),
...headers,
},
body: body ? JSON.stringify(body) : undefined,
});
if (res.status === 204) return null;
const isJson = (res.headers.get("content-type") ?? "").includes("json");
const payload = isJson ? await res.json() : await res.arrayBuffer();
if (!res.ok) {
// RFC 7807 problem: { type, title, status, detail, code, invalid_tasks? }
const err = new Error(payload?.detail || payload?.title || `HTTP ${res.status}`);
err.status = res.status;
err.code = payload?.code;
err.problem = payload;
throw err;
}
return payload;
}
1. Create your jobs
There are several ways to create a job. Pick the one that matches how your URLs arrive.Submit a job with all URLs up front
Create a closed job (one where you know all URLs up front) withPOST /jobs. The response includes a job_id and the initial run status.
POST /jobs returns 201 Created for submissions under 10,000 inline tasks and 202 Accepted for 10,000 or more tasks (or a CSV input). The request shape and response body are identical, so treat both the same: store job_id, then poll until stats.completed >= stats.total (or use a webhook). On a 202, stats.total is correct immediately, but GET /jobs/{id}/results may return partial pages for a few seconds while the task rows finish writing.## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
payload = {
'type': 'regular',
'status': 'closed',
'zenrows_params': {'js_render': 'true', 'premium_proxy': 'true'},
'tasks': [
{'url': 'https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie', 'external_id': 'order-1'},
{'url': 'https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket', 'external_id': 'order-2'},
],
}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.json())
// Reuses the api() helper from the Basics section.
const submitted = await api("POST", "/jobs", {
body: {
type: "regular",
status: "closed",
zenrows_params: { js_render: "true", premium_proxy: "true" },
tasks: [
{ url: "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie", external_id: "order-1" },
{ url: "https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket", external_id: "order-2" },
],
},
});
console.log("submitted", submitted.job_id, submitted.latest_run.status);
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class SubmitJob {
public static void main(final String... args) throws Exception {
String body = "{"
+ "\"type\":\"regular\",\"status\":\"closed\","
+ "\"zenrows_params\":{\"js_render\":\"true\",\"premium_proxy\":\"true\"},"
+ "\"tasks\":["
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie\",\"external_id\":\"order-1\"},"
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket\",\"external_id\":\"order-2\"}"
+ "]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
$payload = json_encode([
'type' => 'regular',
'status' => 'closed',
'zenrows_params' => ['js_render' => 'true', 'premium_proxy' => 'true'],
'tasks' => [
['url' => 'https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie', 'external_id' => 'order-1'],
['url' => 'https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket', 'external_id' => 'order-2'],
],
]);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
echo $response . PHP_EOL;
curl_close($ch);
?>
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "premium_proxy": "true" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie", "external_id": "order-1" },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket", "external_id": "order-2" }
]
}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
# gem install faraday
require 'faraday'
require 'json'
payload = {
type: 'regular',
status: 'closed',
zenrows_params: { js_render: 'true', premium_proxy: 'true' },
tasks: [
{ url: 'https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie', external_id: 'order-1' },
{ url: 'https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket', external_id: 'order-2' }
]
}
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = payload.to_json
end
print(res.body)
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "premium_proxy": "true" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie", "external_id": "order-1" },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket", "external_id": "order-2" }
]
}'
Configure the scrape and per-URL options
zenrows_params applies to every URL in the job. A per-task zenrows_params overrides the job-level keys for that URL, so you can set the common case once and tune exceptions such as a different proxy_country.
Common keys:
js_renderpremium_proxyproxy_countryjs_instructionswait_forwaitblock_resourcescustom_headersjson_responsecss_extractorautoparseresponse_type(markdown,plaintext, orpdf)outputsoriginal_statusallowed_status_codessession_id
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
payload = {
'type': 'regular',
'status': 'closed',
'zenrows_params': {'js_render': 'true', 'proxy_country': 'us', 'response_type': 'markdown'},
'tasks': [
{'url': 'https://www.scrapingcourse.com/ecommerce/product/artemis-running-short',
'external_id': 'order-42', 'zenrows_params': {'proxy_country': 'de'}},
{'url': 'https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee',
'metadata': {'sku': 'ABC-1'}},
],
}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.json())
const job = await api("POST", "/jobs", {
body: {
type: "regular",
status: "closed",
zenrows_params: { js_render: "true", proxy_country: "us", response_type: "markdown" },
tasks: [
{ url: "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short",
external_id: "order-42", zenrows_params: { proxy_country: "de" } },
{ url: "https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee",
metadata: { sku: "ABC-1" } },
],
},
});
console.log(job.job_id);
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class ConfigureJob {
public static void main(final String... args) throws Exception {
String body = "{"
+ "\"type\":\"regular\",\"status\":\"closed\","
+ "\"zenrows_params\":{\"js_render\":\"true\",\"proxy_country\":\"us\",\"response_type\":\"markdown\"},"
+ "\"tasks\":["
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/artemis-running-short\",\"external_id\":\"order-42\",\"zenrows_params\":{\"proxy_country\":\"de\"}},"
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee\",\"metadata\":{\"sku\":\"ABC-1\"}}"
+ "]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
$payload = json_encode([
'type' => 'regular',
'status' => 'closed',
'zenrows_params' => ['js_render' => 'true', 'proxy_country' => 'us', 'response_type' => 'markdown'],
'tasks' => [
['url' => 'https://www.scrapingcourse.com/ecommerce/product/artemis-running-short',
'external_id' => 'order-42', 'zenrows_params' => ['proxy_country' => 'de']],
['url' => 'https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee',
'metadata' => ['sku' => 'ABC-1']],
],
]);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "proxy_country": "us", "response_type": "markdown" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short", "external_id": "order-42", "zenrows_params": { "proxy_country": "de" } },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee", "metadata": { "sku": "ABC-1" } }
]
}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
# gem install faraday
require 'faraday'
require 'json'
payload = {
type: 'regular',
status: 'closed',
zenrows_params: { js_render: 'true', proxy_country: 'us', response_type: 'markdown' },
tasks: [
{ url: 'https://www.scrapingcourse.com/ecommerce/product/artemis-running-short',
external_id: 'order-42', zenrows_params: { proxy_country: 'de' } },
{ url: 'https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee',
metadata: { sku: 'ABC-1' } }
]
}
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = payload.to_json
end
print(res.body)
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "proxy_country": "us", "response_type": "markdown" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short", "external_id": "order-42", "zenrows_params": { "proxy_country": "de" } },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee", "metadata": { "sku": "ABC-1" } }
]
}'
The API returns your
external_id unchanged on every result, so you can match results back to your own records without tracking the API’s task IDs. It does not need to be unique.Open jobs: adding tasks in batches
Open (queue-mode) jobs are still being finalized during the private beta and may not be available yet. If
POST /jobs with status: "open" returns a 503, use a closed job or a CSV upload for now.POST /jobs/{job_id}/tasks over time, then close it by sending last_batch: true on the final batch. Each add-tasks call accepts up to 10,000 URLs.
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
job = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers,
data=json.dumps({'type': 'regular', 'status': 'open',
'zenrows_params': {'js_render': 'true'}})).json()
job_id = job['job_id']
requests.post(f'https://async.api.zenrows.com/v1/jobs/{job_id}/tasks', headers=headers,
data=json.dumps({'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/page/1'}]}))
requests.post(f'https://async.api.zenrows.com/v1/jobs/{job_id}/tasks', headers=headers,
data=json.dumps({'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/page/2'}], 'last_batch': True}))
const { job_id } = await api("POST", "/jobs", {
body: { type: "regular", status: "open", zenrows_params: { js_render: "true" } },
});
await api("POST", `/jobs/${job_id}/tasks`, {
body: { tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/page/1" }] },
});
await api("POST", `/jobs/${job_id}/tasks`, {
body: { tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/page/2" }], last_batch: true }, // closes the job
});
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class OpenJob {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String created = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", key)
.bodyString("{\"type\":\"regular\",\"status\":\"open\",\"zenrows_params\":{\"js_render\":\"true\"}}", ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(created); // read job_id from this response
String jobId = "JOB_ID";
Request.post("https://async.api.zenrows.com/v1/jobs/" + jobId + "/tasks")
.addHeader("X-API-Key", key)
.bodyString("{\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/page/1\"}]}", ContentType.APPLICATION_JSON)
.execute().discardContent();
Request.post("https://async.api.zenrows.com/v1/jobs/" + jobId + "/tasks")
.addHeader("X-API-Key", key)
.bodyString("{\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/page/2\"}],\"last_batch\":true}", ContentType.APPLICATION_JSON)
.execute().discardContent();
}
}
<?php
function post($path, $body) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1' . $path);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($body));
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
curl_close($ch);
return json_decode($res, true);
}
$job = post('/jobs', ['type' => 'regular', 'status' => 'open', 'zenrows_params' => ['js_render' => 'true']]);
$jobId = $job['job_id'];
post("/jobs/$jobId/tasks", ['tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/page/1']]]);
post("/jobs/$jobId/tasks", ['tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/page/2']], 'last_batch' => true]);
?>
package main
import (
"bytes"
"log"
"net/http"
)
func post(path string, body string) {
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1"+path, bytes.NewBufferString(body))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
}
func main() {
// Create the open job, then read job_id from the response body.
post("/jobs", `{"type":"regular","status":"open","zenrows_params":{"js_render":"true"}}`)
jobId := "JOB_ID"
post("/jobs/"+jobId+"/tasks", `{"tasks":[{"url":"https://www.scrapingcourse.com/ecommerce/page/1"}]}`)
post("/jobs/"+jobId+"/tasks", `{"tasks":[{"url":"https://www.scrapingcourse.com/ecommerce/page/2"}],"last_batch":true}`)
}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
conn.headers['Content-Type'] = 'application/json'
job = JSON.parse(conn.post('https://async.api.zenrows.com/v1/jobs',
{ type: 'regular', status: 'open', zenrows_params: { js_render: 'true' } }.to_json).body)
job_id = job['job_id']
conn.post("https://async.api.zenrows.com/v1/jobs/#{job_id}/tasks",
{ tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/page/1' }] }.to_json)
conn.post("https://async.api.zenrows.com/v1/jobs/#{job_id}/tasks",
{ tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/page/2' }], last_batch: true }.to_json)
# 1) Create the open job and note its job_id.
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "open", "zenrows_params": { "js_render": "true" } }'
# 2) Add a batch of tasks.
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/page/1" }] }'
# 3) Add the final batch and close the job.
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/page/2" }], "last_batch": true }'
last_batch: true (or POST /jobs/{id}/close) locks the job, and later adds return 409. To abandon a run that is still in flight, call POST /jobs/{id}/stop. A very large submission also keeps ingesting task rows for a few seconds after it responds; POST /jobs/{id}/tasks can return 409 during that window, so retry shortly.
Large inputs: CSV upload
For lists too large to inline, upload a CSV in three steps:- Create an input slot with
POST /job_inputs. - PUT the file to the returned presigned URL using the exact headers it specifies
- Submit a job referencing
file_input_id.
fields map points the canonical url (required) and external_id (optional) fields at your CSV columns by header name (with header: true) or 0-based index.
Limits: 50 MB / 100,000 rows. Upload slots live 24 hours.
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
# 1) Create the slot.
slot = requests.post('https://async.api.zenrows.com/v1/job_inputs',
headers={**headers, 'Content-Type': 'application/json'},
data=json.dumps({'type': 'csv',
'csv': {'header': True, 'fields': {'url': 'URL', 'external_id': 'Customer Ref'}}})).json()
# 2) Upload the file with the exact content-type the slot requires.
with open('urls.csv', 'rb') as f:
requests.put(slot['upload']['url'], headers=slot['upload']['headers'], data=f)
# 3) Submit referencing the slot.
job = requests.post('https://async.api.zenrows.com/v1/jobs',
headers={**headers, 'Content-Type': 'application/json'},
data=json.dumps({'type': 'regular', 'status': 'closed',
'file_input_id': slot['file_input_id'], 'zenrows_params': {'js_render': 'true'}})).json()
print(job['job_id'])
import { readFile } from "node:fs/promises";
// 1) Create the slot.
const slot = await api("POST", "/job_inputs", {
body: { type: "csv", csv: { header: true, fields: { url: "URL", external_id: "Customer Ref" } } },
});
// 2) Upload the body with the EXACT content-type the slot requires.
const csvBytes = await readFile("urls.csv");
const put = await fetch(slot.upload.url, {
method: slot.upload.method, // "PUT"
headers: slot.upload.headers, // e.g. { "Content-Type": "text/csv" }
body: csvBytes,
});
if (!put.ok) throw new Error(`upload failed: ${put.status}`);
// 3) Submit referencing the slot instead of inline tasks.
const job = await api("POST", "/jobs", {
body: { type: "regular", status: "closed", file_input_id: slot.file_input_id, zenrows_params: { js_render: "true" } },
});
console.log(job.job_id);
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
import java.io.File;
public class CsvUpload {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
// 1) Create the slot, then read upload.url, upload.headers, and file_input_id from the response.
String slot = Request.post("https://async.api.zenrows.com/v1/job_inputs")
.addHeader("X-API-Key", key)
.bodyString("{\"type\":\"csv\",\"csv\":{\"header\":true,\"fields\":{\"url\":\"URL\",\"external_id\":\"Customer Ref\"}}}", ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(slot);
// 2) PUT the file to the presigned URL with the content-type the slot requires.
Request.put("PRESIGNED_UPLOAD_URL")
.bodyFile(new File("urls.csv"), ContentType.create("text/csv"))
.execute().discardContent();
// 3) Submit referencing the slot.
String job = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", key)
.bodyString("{\"type\":\"regular\",\"status\":\"closed\",\"file_input_id\":\"FILE_INPUT_ID\",\"zenrows_params\":{\"js_render\":\"true\"}}", ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(job);
}
}
<?php
$key = 'YOUR_ZENROWS_API_KEY';
// 1) Create the slot.
$ch = curl_init('https://async.api.zenrows.com/v1/job_inputs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode([
'type' => 'csv',
'csv' => ['header' => true, 'fields' => ['url' => 'URL', 'external_id' => 'Customer Ref']],
]));
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$slot = json_decode(curl_exec($ch), true);
curl_close($ch);
// 2) Upload the file with the exact content-type the slot requires.
$ch = curl_init($slot['upload']['url']);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_POSTFIELDS, file_get_contents('urls.csv'));
curl_setopt($ch, CURLOPT_HTTPHEADER, $slot['upload']['headers']);
curl_exec($ch);
curl_close($ch);
// 3) Submit referencing the slot.
$ch = curl_init('https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode([
'type' => 'regular', 'status' => 'closed',
'file_input_id' => $slot['file_input_id'], 'zenrows_params' => ['js_render' => 'true'],
]));
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"bytes"
"io"
"log"
"net/http"
"os"
)
func main() {
key := "YOUR_ZENROWS_API_KEY"
// 1) Create the slot, then read upload.url, upload.headers, and file_input_id from the response.
slotReq, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/job_inputs",
bytes.NewBufferString(`{"type":"csv","csv":{"header":true,"fields":{"url":"URL","external_id":"Customer Ref"}}}`))
slotReq.Header.Set("X-API-Key", key)
slotReq.Header.Set("Content-Type", "application/json")
slotResp, err := http.DefaultClient.Do(slotReq)
if err != nil {
log.Fatalln(err)
}
defer slotResp.Body.Close()
slotBody, _ := io.ReadAll(slotResp.Body)
log.Println(string(slotBody))
// 2) PUT the file to the presigned URL with the content-type the slot requires.
file, _ := os.Open("urls.csv")
defer file.Close()
putReq, _ := http.NewRequest("PUT", "PRESIGNED_UPLOAD_URL", file)
putReq.Header.Set("Content-Type", "text/csv")
if _, err := http.DefaultClient.Do(putReq); err != nil {
log.Fatalln(err)
}
// 3) Submit referencing the slot.
jobReq, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs",
bytes.NewBufferString(`{"type":"regular","status":"closed","file_input_id":"FILE_INPUT_ID","zenrows_params":{"js_render":"true"}}`))
jobReq.Header.Set("X-API-Key", key)
jobReq.Header.Set("Content-Type", "application/json")
jobResp, err := http.DefaultClient.Do(jobReq)
if err != nil {
log.Fatalln(err)
}
defer jobResp.Body.Close()
jobBody, _ := io.ReadAll(jobResp.Body)
log.Println(string(jobBody))
}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
conn = Faraday.new
# 1) Create the slot.
slot = JSON.parse(conn.post('https://async.api.zenrows.com/v1/job_inputs',
{ type: 'csv', csv: { header: true, fields: { url: 'URL', external_id: 'Customer Ref' } } }.to_json,
{ 'X-API-Key' => key, 'Content-Type' => 'application/json' }).body)
# 2) Upload the file with the exact headers the slot requires.
conn.put(slot['upload']['url'], File.read('urls.csv'), slot['upload']['headers'])
# 3) Submit referencing the slot.
job = JSON.parse(conn.post('https://async.api.zenrows.com/v1/jobs',
{ type: 'regular', status: 'closed', file_input_id: slot['file_input_id'], zenrows_params: { js_render: 'true' } }.to_json,
{ 'X-API-Key' => key, 'Content-Type' => 'application/json' }).body)
puts job['job_id']
# 1) Create the slot. Read upload.url, upload.headers, and file_input_id from the response.
curl -X POST "https://async.api.zenrows.com/v1/job_inputs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "csv", "csv": { "header": true, "fields": { "url": "URL", "external_id": "Customer Ref" } } }'
# 2) Upload the file to the presigned URL with the content-type it requires.
curl -X PUT "PRESIGNED_UPLOAD_URL" -H "Content-Type: text/csv" --data-binary @urls.csv
# 3) Submit referencing the slot.
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "closed", "file_input_id": "FILE_INPUT_ID", "zenrows_params": { "js_render": "true" } }'
Scheduled jobs
A scheduled job stores its URLs as a template, and each fire creates a new run. Provide exactly one ofat, rate, or calendar (plus a timezone for at and calendar). Times of day must be full hours. The request below schedules a job to run every 6 hours.
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
payload = {
'type': 'scheduled', 'status': 'closed',
'schedule': {'rate': {'every': 6, 'unit': 'hour'}},
'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/'}],
}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.json())
const job = await api("POST", "/jobs", {
body: {
type: "scheduled", status: "closed",
schedule: { rate: { every: 6, unit: "hour" } },
tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/" }],
},
});
console.log(job.job_id);
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class ScheduledJob {
public static void main(final String... args) throws Exception {
String body = "{\"type\":\"scheduled\",\"status\":\"closed\","
+ "\"schedule\":{\"rate\":{\"every\":6,\"unit\":\"hour\"}},"
+ "\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/\"}]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
$payload = json_encode([
'type' => 'scheduled', 'status' => 'closed',
'schedule' => ['rate' => ['every' => 6, 'unit' => 'hour']],
'tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/']],
]);
$ch = curl_init('https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{
"type": "scheduled", "status": "closed",
"schedule": { "rate": { "every": 6, "unit": "hour" } },
"tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/" }]
}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
# gem install faraday
require 'faraday'
require 'json'
payload = {
type: 'scheduled', status: 'closed',
schedule: { rate: { every: 6, unit: 'hour' } },
tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/' }]
}
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = payload.to_json
end
print(res.body)
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{
"type": "scheduled", "status": "closed",
"schedule": { "rate": { "every": 6, "unit": "hour" } },
"tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/" }]
}'
schedule object. The example below runs at 09:00 and 18:00 on Mon/Wed/Fri in Berlin time. Use an at object instead for a one-shot run at a single wall-clock time.
{
"schedule": {
"calendar": {
"times_of_day": ["09:00", "18:00"],
"cadence": { "weekly": { "days": ["mon", "wed", "fri"] } }
},
"timezone": "Europe/Berlin"
}
}
PUT /jobs/{id}/schedule to replace it, and POST /jobs/{id}/schedule/state with {"schedule_state": "paused"} or {"schedule_state": "active"} to pause or resume.
2. Track your jobs
Check progress and find your jobs once they are running.Check job status
Read a job withGET /jobs/{job_id}. Progress lives in latest_run.stats, where completed equals successful plus failed.
## pip install requests
import requests
response = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'})
job = response.json()
print(job['latest_run']['status'], job['latest_run']['stats'])
const job = await api("GET", "/jobs/JOB_ID");
console.log(job.latest_run.status, job.latest_run.stats);
import org.apache.hc.client5.http.fluent.Request;
public class GetJob {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs/JOB_ID');
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
print(res.body)
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
latest_run.status reaches a terminal state (completed, stopped, or deleted), backing off between calls rather than running a tight loop.
## pip install requests
import time, requests
def wait_for_job(job_id, timeout=600):
terminal = {'completed', 'stopped', 'deleted'}
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
deadline = time.time() + timeout
delay = 2.0
while True:
job = requests.get(f'https://async.api.zenrows.com/v1/jobs/{job_id}', headers=headers).json()
run = job.get('latest_run')
if run and run['status'] in terminal:
return job
if time.time() > deadline:
raise TimeoutError('timeout waiting for job')
time.sleep(delay)
delay = min(delay * 1.5, 15.0) # back off gradually, capped at 15s
job = wait_for_job('JOB_ID')
s = job['latest_run']['stats'] # { total, completed, successful, failed }
print(job['latest_run']['status'], f"{s['successful']}/{s['total']} successful")
async function waitForJob(jobId, { timeoutMs = 600_000 } = {}) {
const TERMINAL = new Set(["completed", "stopped", "deleted"]);
const deadline = Date.now() + timeoutMs;
let delay = 2000;
for (;;) {
const job = await api("GET", `/jobs/${jobId}`);
const run = job.latest_run;
if (run && TERMINAL.has(run.status)) return job;
if (Date.now() > deadline) throw new Error("timeout waiting for job");
await new Promise((r) => setTimeout(r, delay));
delay = Math.min(delay * 1.5, 15_000); // back off gradually, capped at 15s
}
}
const job = await waitForJob("JOB_ID");
const s = job.latest_run.stats; // { total, completed, successful, failed }
console.log(`${job.latest_run.status}: ${s.successful}/${s.total} successful`);
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import java.util.Set;
public class WaitForJob {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
Set<String> terminal = Set.of("completed", "stopped", "deleted");
long deadline = System.currentTimeMillis() + 600_000;
long delay = 2000;
while (true) {
String body = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
JsonNode run = M.readTree(body).path("latest_run");
if (terminal.contains(run.path("status").asText())) {
System.out.println(run.path("stats"));
break;
}
if (System.currentTimeMillis() > deadline) throw new RuntimeException("timeout waiting for job");
Thread.sleep(delay);
delay = Math.min((long) (delay * 1.5), 15_000); // back off gradually, capped at 15s
}
}
}
<?php
function get_job($id) {
$ch = curl_init("https://async.api.zenrows.com/v1/jobs/$id");
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
curl_close($ch);
return json_decode($res, true);
}
$terminal = ['completed', 'stopped', 'deleted'];
$deadline = time() + 600;
$delay = 2.0;
while (true) {
$job = get_job('JOB_ID');
$run = $job['latest_run'] ?? null;
if ($run && in_array($run['status'], $terminal, true)) { print_r($run['stats']); break; }
if (time() > $deadline) throw new RuntimeException('timeout waiting for job');
usleep((int)($delay * 1000000));
$delay = min($delay * 1.5, 15.0); // back off gradually, capped at 15s
}
?>
package main
import (
"encoding/json"
"io"
"log"
"net/http"
"time"
)
func getJob(id string) map[string]any {
req, _ := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/"+id, nil)
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var job map[string]any
json.Unmarshal(body, &job)
return job
}
func main() {
terminal := map[string]bool{"completed": true, "stopped": true, "deleted": true}
deadline := time.Now().Add(10 * time.Minute)
delay := 2 * time.Second
for {
job := getJob("JOB_ID")
run, _ := job["latest_run"].(map[string]any)
status, _ := run["status"].(string)
if run != nil && terminal[status] {
log.Println(run["stats"])
break
}
if time.Now().After(deadline) {
log.Fatalln("timeout waiting for job")
}
time.Sleep(delay)
if delay < 15*time.Second {
delay = time.Duration(float64(delay) * 1.5)
}
}
}
# gem install faraday
require 'faraday'
require 'json'
def get_job(id)
res = Faraday.get("https://async.api.zenrows.com/v1/jobs/#{id}") do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
JSON.parse(res.body)
end
terminal = ['completed', 'stopped', 'deleted']
deadline = Time.now + 600
delay = 2.0
loop do
job = get_job('JOB_ID')
run = job['latest_run']
if run && terminal.include?(run['status'])
p run['stats']
break
end
raise 'timeout waiting for job' if Time.now > deadline
sleep(delay)
delay = [delay * 1.5, 15.0].min # back off gradually, capped at 15s
end
# cURL has no loop of its own; poll with a small shell loop until the run is terminal.
delay=2
while :; do
status=$(curl -s "https://async.api.zenrows.com/v1/jobs/JOB_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" | jq -r '.latest_run.status')
echo "status: $status"
case "$status" in completed|stopped|deleted) break ;; esac
sleep "$delay"
delay=$(( delay < 15 ? delay + delay/2 + 1 : 15 )) # back off, capped at 15s
done
List and paginate jobs
List your jobs withGET /jobs, optionally filtered by status and type, and page with cursor until next_cursor is null.
## pip install requests
import requests
response = requests.get('https://async.api.zenrows.com/v1/jobs',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'},
params={'status': 'closed', 'type': 'regular'})
print(response.json())
const page = await api("GET", "/jobs", { query: { status: "closed", type: "regular" } });
console.log(page.jobs);
import org.apache.hc.client5.http.fluent.Request;
public class ListJobs {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs?status=closed&type=regular")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs?status=closed&type=regular');
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.params['status'] = 'closed'
req.params['type'] = 'regular'
end
print(res.body)
curl "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
GET /jobs/{id}/runs pages identically.
## pip install requests
import requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
cursor = None
while True:
page = requests.get('https://async.api.zenrows.com/v1/jobs', headers=headers,
params={'status': 'closed', 'type': 'regular', 'cursor': cursor}).json()
for job in page['jobs']:
latest = job.get('latest_run')
print(job['job_id'], job['status'], latest['stats']['total'] if latest else 0)
cursor = page['next_cursor']
if not cursor:
break
async function* iterJobs({ status, type } = {}) {
let cursor;
do {
const page = await api("GET", "/jobs", { query: { status, type, cursor } });
yield* page.jobs;
cursor = page.next_cursor;
} while (cursor);
}
for await (const job of iterJobs({ status: "closed", type: "regular" })) {
console.log(job.job_id, job.status, job.latest_run?.stats.total ?? 0);
}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
public class ListJobsPaged {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String cursor = null;
do {
String url = "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular"
+ (cursor != null ? "&cursor=" + cursor : "");
JsonNode page = M.readTree(Request.get(url).addHeader("X-API-Key", key)
.execute().returnContent().asString());
for (JsonNode job : page.path("jobs")) {
long total = job.path("latest_run").path("stats").path("total").asLong(0);
System.out.println(job.path("job_id").asText() + " " + job.path("status").asText() + " " + total);
}
cursor = page.path("next_cursor").asText(null);
} while (cursor != null && !cursor.isEmpty());
}
}
<?php
$key = 'YOUR_ZENROWS_API_KEY';
$cursor = null;
do {
$url = 'https://async.api.zenrows.com/v1/jobs?status=closed&type=regular'
. ($cursor ? '&cursor=' . urlencode($cursor) : '');
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key"]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$page = json_decode(curl_exec($ch), true);
curl_close($ch);
foreach ($page['jobs'] as $job) {
$total = $job['latest_run']['stats']['total'] ?? 0;
echo "{$job['job_id']} {$job['status']} $total" . PHP_EOL;
}
$cursor = $page['next_cursor'] ?? null;
} while ($cursor);
?>
package main
import (
"encoding/json"
"io"
"log"
"net/http"
)
func main() {
key := "YOUR_ZENROWS_API_KEY"
cursor := ""
for {
url := "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular"
if cursor != "" {
url += "&cursor=" + cursor
}
req, _ := http.NewRequest("GET", url, nil)
req.Header.Set("X-API-Key", key)
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var page struct {
Jobs []struct {
JobID string `json:"job_id"`
Status string `json:"status"`
LatestRun *struct {
Stats struct {
Total int `json:"total"`
} `json:"stats"`
} `json:"latest_run"`
} `json:"jobs"`
NextCursor string `json:"next_cursor"`
}
json.Unmarshal(body, &page)
for _, job := range page.Jobs {
total := 0
if job.LatestRun != nil {
total = job.LatestRun.Stats.Total
}
log.Println(job.JobID, job.Status, total)
}
cursor = page.NextCursor
if cursor == "" {
break
}
}
}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
cursor = nil
loop do
res = Faraday.get('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = key
req.params['status'] = 'closed'
req.params['type'] = 'regular'
req.params['cursor'] = cursor if cursor
end
page = JSON.parse(res.body)
page['jobs'].each do |job|
total = job.dig('latest_run', 'stats', 'total') || 0
puts "#{job['job_id']} #{job['status']} #{total}"
end
cursor = page['next_cursor']
break unless cursor
end
cursor=""
while :; do
page=$(curl -s "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular&cursor=$cursor" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY")
echo "$page" | jq -r '.jobs[] | [.job_id, .status, (.latest_run.stats.total // 0)] | @tsv'
cursor=$(echo "$page" | jq -r '.next_cursor // empty')
[ -z "$cursor" ] && break
done
3. Collect your results
Retrieve the scraped content once a run completes.Download content
List a run’s results withGET /jobs/{job_id}/results, filtered by status (successful, failed, or all) and paged with the cursor query parameter until next_cursor is null.
Each row’s
result_url is a presigned link valid for 2 hours, so fetch it promptly.## pip install requests
import requests
response = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'},
params={'status': 'successful'})
page = response.json()
for row in page['results']:
print(row.get('external_id') or row['task_id'], '->', row['result_url'])
print('next_cursor:', page['next_cursor'])
const page = await api("GET", "/jobs/JOB_ID/results", { query: { status: "successful" } });
for (const row of page.results) {
console.log(row.external_id ?? row.task_id, "->", row.result_url);
}
console.log("next_cursor:", page.next_cursor);
import org.apache.hc.client5.http.fluent.Request;
public class GetResults {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful');
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.params['status'] = 'successful'
end
print(res.body)
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
next_cursor is null, fetching each presigned result_url promptly.
## pip install requests
import os, requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
os.makedirs('out', exist_ok=True)
cursor = None
while True:
page = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results',
headers=headers, params={'status': 'successful', 'cursor': cursor}).json()
for row in page['results']:
# result_url is a presigned link, valid 2h, so fetch it promptly.
content = requests.get(row['result_url']).content
with open(f"out/{row.get('external_id') or row['task_id']}", 'wb') as f:
f.write(content)
cursor = page['next_cursor']
if not cursor:
break
async function* iterResults(jobId, status = "all") {
let cursor;
do {
const page = await api("GET", `/jobs/${jobId}/results`, { query: { status, cursor } });
yield* page.results;
cursor = page.next_cursor;
} while (cursor);
}
import { writeFile } from "node:fs/promises";
for await (const row of iterResults("JOB_ID", "successful")) {
// row.result_url is a presigned link, valid 2h, so fetch it promptly.
const bytes = Buffer.from(await (await fetch(row.result_url)).arrayBuffer());
await writeFile(`./out/${row.external_id ?? row.task_id}`, bytes);
}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import java.nio.file.Files;
import java.nio.file.Path;
public class DownloadResults {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String cursor = null;
do {
String url = "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful"
+ (cursor != null ? "&cursor=" + cursor : "");
JsonNode page = M.readTree(Request.get(url).addHeader("X-API-Key", key)
.execute().returnContent().asString());
for (JsonNode row : page.path("results")) {
// result_url is a presigned link, valid 2h, so fetch it promptly.
byte[] bytes = Request.get(row.path("result_url").asText())
.execute().returnContent().asBytes();
String name = row.path("external_id").asText(row.path("task_id").asText());
Files.write(Path.of("out", name), bytes);
}
cursor = page.path("next_cursor").asText(null);
} while (cursor != null && !cursor.isEmpty());
}
}
<?php
$key = 'YOUR_ZENROWS_API_KEY';
@mkdir('out');
$cursor = null;
do {
$url = 'https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful'
. ($cursor ? '&cursor=' . urlencode($cursor) : '');
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key"]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$page = json_decode(curl_exec($ch), true);
curl_close($ch);
foreach ($page['results'] as $row) {
// result_url is a presigned link, valid 2h, so fetch it promptly.
$name = $row['external_id'] ?? $row['task_id'];
file_put_contents("out/$name", file_get_contents($row['result_url']));
}
$cursor = $page['next_cursor'] ?? null;
} while ($cursor);
?>
package main
import (
"encoding/json"
"io"
"log"
"net/http"
"os"
"path/filepath"
)
func get(url, key string) []byte {
req, _ := http.NewRequest("GET", url, nil)
if key != "" {
req.Header.Set("X-API-Key", key)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return body
}
func main() {
key := "YOUR_ZENROWS_API_KEY"
os.MkdirAll("out", 0o755)
cursor := ""
for {
url := "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful"
if cursor != "" {
url += "&cursor=" + cursor
}
var page struct {
Results []struct {
TaskID string `json:"task_id"`
ExternalID string `json:"external_id"`
ResultURL string `json:"result_url"`
} `json:"results"`
NextCursor string `json:"next_cursor"`
}
json.Unmarshal(get(url, key), &page)
for _, row := range page.Results {
// result_url is a presigned link, valid 2h, so fetch it promptly.
name := row.ExternalID
if name == "" {
name = row.TaskID
}
os.WriteFile(filepath.Join("out", name), get(row.ResultURL, ""), 0o644)
}
cursor = page.NextCursor
if cursor == "" {
break
}
}
}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
Dir.mkdir('out') unless Dir.exist?('out')
cursor = nil
loop do
res = Faraday.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results') do |req|
req.headers['X-API-Key'] = key
req.params['status'] = 'successful'
req.params['cursor'] = cursor if cursor
end
page = JSON.parse(res.body)
page['results'].each do |row|
# result_url is a presigned link, valid 2h, so fetch it promptly.
name = row['external_id'] || row['task_id']
File.write("out/#{name}", Faraday.get(row['result_url']).body, mode: 'wb')
end
cursor = page['next_cursor']
break unless cursor
end
# Page with a shell loop, then download each presigned result_url.
mkdir -p out
cursor=""
while :; do
page=$(curl -s "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful&cursor=$cursor" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY")
echo "$page" | jq -r '.results[] | [(.external_id // .task_id), .result_url] | @tsv' \
| while IFS=$'\t' read -r name url; do curl -s "$url" -o "out/$name"; done
cursor=$(echo "$page" | jq -r '.next_cursor // empty')
[ -z "$cursor" ] && break
done
GET /jobs/{job_id}/tasks/{task_id}/content.
200: returns the content409: means the task is still processing422: means the task failed and the body holds the stored error.
## pip install requests
import requests
# 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
response = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'})
print(response.status_code)
print(response.text)
// 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
const content = await api("GET", "/jobs/JOB_ID/tasks/TASK_ID/content");
import org.apache.hc.client5.http.fluent.Request;
public class TaskContent {
public static void main(final String... args) throws Exception {
// 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
// 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content');
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
echo curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL;
echo $response . PHP_EOL;
curl_close($ch);
?>
package main
import (
"io"
"log"
"net/http"
)
func main() {
// 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(resp.StatusCode, string(body))
}
# gem install faraday
require 'faraday'
# 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
print(res.status)
print(res.body)
# 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
curl -i "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
A failed task’s results row already carries
row.error ({ code, detail, ... }), so you rarely need the content endpoint to inspect a failure. During the beta, treat the error code as a hint: it does not yet reliably distinguish blocked, not-found, and timeout.Bulk export (ZIP)
Bundle an entire run’s result bodies into a single ZIP. Start the export withPOST /jobs/{job_id}/runs/{run_id}/exports, poll the export until status is completed, then download the presigned download_url.
Exports expire 12 hours after creation, the API presigns
download_url fresh on each poll, and a run whose combined results exceed 1 GB fails with a clear error instead of producing a partial archive.## pip install requests
import requests
started = requests.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'}).json()
print(started['export_id'], started['status'])
# Poll the export by id until status == 'completed', then download started['download_url'].
const started = await api("POST", "/jobs/JOB_ID/runs/RUN_ID/exports");
console.log(started.export_id, started.status);
// Poll the export by id until status === 'completed', then download download_url.
import org.apache.hc.client5.http.fluent.Request;
public class StartExport {
public static void main(final String... args) throws Exception {
String response = Request.post("https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
// Poll the export by id until status is "completed", then download download_url.
}
}
<?php
$ch = curl_init('https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
// Poll the export by id until status is "completed", then download download_url.
?>
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
// Poll the export by id until status is "completed", then download download_url.
}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
print(res.body)
# Poll the export by id until status is "completed", then download download_url.
# Start the export, then poll the returned export_id until status is "completed".
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports/EXPORT_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
completed, then download the ZIP.
## pip install requests
import time, requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
base = 'https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports'
exp = requests.post(base, headers=headers).json()
while exp['status'] not in ('completed', 'failed'):
time.sleep(3)
exp = requests.get(f"{base}/{exp['export_id']}", headers=headers).json()
if exp['status'] == 'failed':
raise RuntimeError(exp.get('error'))
with open('results.zip', 'wb') as f:
f.write(requests.get(exp['download_url']).content)
import { writeFile } from "node:fs/promises";
let exp = await api("POST", "/jobs/JOB_ID/runs/RUN_ID/exports");
while (exp.status !== "completed" && exp.status !== "failed") {
await new Promise((r) => setTimeout(r, 3000));
exp = await api("GET", `/jobs/JOB_ID/runs/RUN_ID/exports/${exp.export_id}`);
}
if (exp.status === "failed") throw new Error(exp.error);
const zip = Buffer.from(await (await fetch(exp.download_url)).arrayBuffer());
await writeFile("results.zip", zip);
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import java.nio.file.Files;
import java.nio.file.Path;
public class ExportRun {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String base = "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports";
JsonNode exp = M.readTree(Request.post(base).addHeader("X-API-Key", key)
.execute().returnContent().asString());
String id = exp.path("export_id").asText();
while (!exp.path("status").asText().equals("completed") && !exp.path("status").asText().equals("failed")) {
Thread.sleep(3000);
exp = M.readTree(Request.get(base + "/" + id).addHeader("X-API-Key", key)
.execute().returnContent().asString());
}
if (exp.path("status").asText().equals("failed")) throw new RuntimeException(exp.path("error").asText());
byte[] zip = Request.get(exp.path("download_url").asText()).execute().returnContent().asBytes();
Files.write(Path.of("results.zip"), zip);
}
}
<?php
$key = 'YOUR_ZENROWS_API_KEY';
$base = 'https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports';
function call($method, $url, $key) {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key"]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
curl_close($ch);
return json_decode($res, true);
}
$exp = call('POST', $base, $key);
while (!in_array($exp['status'], ['completed', 'failed'], true)) {
sleep(3);
$exp = call('GET', "$base/{$exp['export_id']}", $key);
}
if ($exp['status'] === 'failed') throw new RuntimeException($exp['error'] ?? 'export failed');
file_put_contents('results.zip', file_get_contents($exp['download_url']));
?>
package main
import (
"encoding/json"
"io"
"log"
"net/http"
"os"
"time"
)
func call(method, url, key string) map[string]any {
req, _ := http.NewRequest(method, url, nil)
if key != "" {
req.Header.Set("X-API-Key", key)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var out map[string]any
json.Unmarshal(body, &out)
return out
}
func main() {
key := "YOUR_ZENROWS_API_KEY"
base := "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports"
exp := call("POST", base, key)
id, _ := exp["export_id"].(string)
for exp["status"] != "completed" && exp["status"] != "failed" {
time.Sleep(3 * time.Second)
exp = call("GET", base+"/"+id, key)
}
if exp["status"] == "failed" {
log.Fatalln(exp["error"])
}
req, _ := http.NewRequest("GET", exp["download_url"].(string), nil)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
out, _ := os.Create("results.zip")
defer out.Close()
io.Copy(out, resp.Body)
}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
base = 'https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports'
exp = JSON.parse(Faraday.post(base) { |r| r.headers['X-API-Key'] = key }.body)
until %w[completed failed].include?(exp['status'])
sleep 3
exp = JSON.parse(Faraday.get("#{base}/#{exp['export_id']}") { |r| r.headers['X-API-Key'] = key }.body)
end
raise(exp['error'] || 'export failed') if exp['status'] == 'failed'
File.write('results.zip', Faraday.get(exp['download_url']).body, mode: 'wb')
base="https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports"
started=$(curl -s -X POST "$base" -H "X-API-Key: YOUR_ZENROWS_API_KEY")
id=$(echo "$started" | jq -r '.export_id')
while :; do
exp=$(curl -s "$base/$id" -H "X-API-Key: YOUR_ZENROWS_API_KEY")
status=$(echo "$exp" | jq -r '.status')
case "$status" in completed|failed) break ;; esac
sleep 3
done
[ "$status" = "failed" ] && { echo "export failed"; exit 1; }
curl -s "$(echo "$exp" | jq -r '.download_url')" -o results.zip
4. Webhooks and notifications
Get notified when a run finishes, and secure those callbacks.Set up webhooks
Attach a webhook so the API POSTs arun.completed event to your HTTPS URL when a run finishes. Set signature: true for HMAC-signed deliveries (see Manage HMAC keys).
You can attach it on the submit body, replace it later with PUT /jobs/{id}/webhook, remove it with DELETE /jobs/{id}/webhook, and probe a receiver with POST /webhook/test.
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
webhook = {'url': 'https://your-app.com/webhooks/zenrows', 'signature': True}
# Set or replace the webhook on an existing job:
requests.put('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook', headers=headers, data=json.dumps(webhook))
# Probe a receiver before wiring it up:
probe = requests.post('https://async.api.zenrows.com/v1/webhook/test', headers=headers, data=json.dumps(webhook)).json()
print(probe['delivered'], probe['status_code'])
# Stop deliveries:
requests.delete('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook', headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'})
const webhook = { url: "https://your-app.com/webhooks/zenrows", signature: true };
// At submit time, include it on the body: { ..., webhook }.
// Or manage it on an existing job:
await api("PUT", "/jobs/JOB_ID/webhook", { body: webhook });
await api("DELETE", "/jobs/JOB_ID/webhook"); // stop deliveries
// Verify a receiver before wiring it to a job:
const probe = await api("POST", "/webhook/test", { body: webhook });
console.log(probe.delivered, probe.status_code);
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class Webhook {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String webhook = "{\"url\":\"https://your-app.com/webhooks/zenrows\",\"signature\":true}";
// Set or replace the webhook on an existing job:
Request.put("https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook")
.addHeader("X-API-Key", key).bodyString(webhook, ContentType.APPLICATION_JSON)
.execute().discardContent();
// Probe a receiver:
String probe = Request.post("https://async.api.zenrows.com/v1/webhook/test")
.addHeader("X-API-Key", key).bodyString(webhook, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(probe);
// Stop deliveries:
Request.delete("https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook")
.addHeader("X-API-Key", key).execute().discardContent();
}
}
<?php
$key = 'YOUR_ZENROWS_API_KEY';
$webhook = json_encode(['url' => 'https://your-app.com/webhooks/zenrows', 'signature' => true]);
// Set or replace the webhook on an existing job:
$ch = curl_init('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook');
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_POSTFIELDS, $webhook);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_exec($ch);
curl_close($ch);
// Probe a receiver:
$ch = curl_init('https://async.api.zenrows.com/v1/webhook/test');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $webhook);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
webhook := `{ "url": "https://your-app.com/webhooks/zenrows", "signature": true }`
// Set or replace the webhook on an existing job:
putReq, _ := http.NewRequest("PUT", "https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook", bytes.NewBufferString(webhook))
putReq.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
putReq.Header.Set("Content-Type", "application/json")
if _, err := http.DefaultClient.Do(putReq); err != nil {
log.Fatalln(err)
}
// Probe a receiver:
testReq, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/webhook/test", bytes.NewBufferString(webhook))
testReq.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
testReq.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(testReq)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
conn.headers['Content-Type'] = 'application/json'
webhook = { url: 'https://your-app.com/webhooks/zenrows', signature: true }.to_json
# Set or replace the webhook on an existing job:
conn.put('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook', webhook)
# Probe a receiver:
puts conn.post('https://async.api.zenrows.com/v1/webhook/test', webhook).body
# Stop deliveries:
conn.delete('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook')
# Set or replace the webhook on an existing job:
curl -X PUT "https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "url": "https://your-app.com/webhooks/zenrows", "signature": true }'
# Probe a receiver:
curl -X POST "https://async.api.zenrows.com/v1/webhook/test" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "url": "https://your-app.com/webhooks/zenrows", "signature": false }'
# Stop deliveries:
curl -X DELETE "https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
The API delivers each event at least once, so deduplicate on the
X-ZenRows-Event-Id header. Webhooks are a convenience; the run state and result URLs remain the reliable source of truth.Manage HMAC keys
Signed webhook deliveries need an active HMAC key for your organization. Rotation uses two slots, an active key plus a candidate, so you can roll keys without downtime.POST /hmac/keys/rotatecreates the first key or stages a candidate (the secret is returned once)POST /hmac/keys/rotate/finalizepromotes the candidate and retires the old keyDELETE /hmac/keys/rotatediscards a staged candidateGET /hmac/keyslists key metadata without secrets
## pip install requests
import requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
created = requests.post('https://async.api.zenrows.com/v1/hmac/keys/rotate', headers=headers).json()
print(created) # store created['secret'] now; it is returned only once
requests.post('https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize', headers=headers)
# requests.delete('https://async.api.zenrows.com/v1/hmac/keys/rotate', headers=headers) # discard candidate
print(requests.get('https://async.api.zenrows.com/v1/hmac/keys', headers=headers).json())
const created = await api("POST", "/hmac/keys/rotate");
console.log(created.kid, created.secret); // the secret is returned ONCE, so store it now
await api("POST", "/hmac/keys/rotate/finalize"); // promote candidate -> active, retire the old
// await api("DELETE", "/hmac/keys/rotate"); // or discard the candidate
console.log(await api("GET", "/hmac/keys")); // metadata only; never returns secrets
import org.apache.hc.client5.http.fluent.Request;
public class HmacKeys {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String created = Request.post("https://async.api.zenrows.com/v1/hmac/keys/rotate")
.addHeader("X-API-Key", key).execute().returnContent().asString();
System.out.println(created); // store the secret now; it is returned only once
Request.post("https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize")
.addHeader("X-API-Key", key).execute().discardContent();
// Request.delete("https://async.api.zenrows.com/v1/hmac/keys/rotate")
// .addHeader("X-API-Key", key).execute().discardContent(); // discard candidate
String list = Request.get("https://async.api.zenrows.com/v1/hmac/keys")
.addHeader("X-API-Key", key).execute().returnContent().asString();
System.out.println(list);
}
}
<?php
$key = 'YOUR_ZENROWS_API_KEY';
function call($method, $path, $key) {
$ch = curl_init('https://async.api.zenrows.com/v1' . $path);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key"]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
curl_close($ch);
return $res;
}
echo call('POST', '/hmac/keys/rotate', $key) . PHP_EOL; // store the secret now
call('POST', '/hmac/keys/rotate/finalize', $key); // promote the candidate
// call('DELETE', '/hmac/keys/rotate', $key); // or discard the candidate
echo call('GET', '/hmac/keys', $key) . PHP_EOL; // metadata only
?>
package main
import (
"io"
"log"
"net/http"
)
func call(method, path string) string {
req, _ := http.NewRequest(method, "https://async.api.zenrows.com/v1"+path, nil)
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return string(body)
}
func main() {
log.Println(call("POST", "/hmac/keys/rotate")) // store the secret now
call("POST", "/hmac/keys/rotate/finalize") // promote the candidate
// call("DELETE", "/hmac/keys/rotate") // or discard the candidate
log.Println(call("GET", "/hmac/keys")) // metadata only
}
# gem install faraday
require 'faraday'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
puts conn.post('https://async.api.zenrows.com/v1/hmac/keys/rotate').body # store the secret now
conn.post('https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize') # promote the candidate
# conn.delete('https://async.api.zenrows.com/v1/hmac/keys/rotate') # or discard the candidate
puts conn.get('https://async.api.zenrows.com/v1/hmac/keys').body # metadata only
# Create the active key or stage a candidate (the secret is returned once):
curl -X POST "https://async.api.zenrows.com/v1/hmac/keys/rotate" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
# Promote the candidate to active and retire the old key:
curl -X POST "https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
# Or discard the staged candidate:
curl -X DELETE "https://async.api.zenrows.com/v1/hmac/keys/rotate" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
# List key metadata (never returns secrets):
curl "https://async.api.zenrows.com/v1/hmac/keys" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
Verify signed webhooks
Whensignature: true, each delivery includes an X-Signature header in the form t=<unix>,v1=<hex>,kid=<active>, where t is the timestamp, v1 is the signature, and kid identifies the key used.
To verify a delivery:
- Look up the base64 secret that matches the delivery’s
kid. - Compute
HMAC-SHA256(secret, "<t>.<raw_body>"), signing the timestamp, a literal dot, and the raw request body. - Compare the result to
v1. If they match, the delivery is authentic. Always sign the raw request body, never a re-serialized version, or the signature will not match. Ifsignatureisfalse, noX-Signatureheader is sent, so authenticate the callback another way, such as a secret path in the URL. Either way, deduplicate on theX-ZenRows-Event-Idheader, since the same delivery can arrive more than once.
v1 in your receiver.
import hmac, hashlib, base64
def verify(raw_body: bytes, t: str, v1: str, secret_b64: str) -> bool:
secret = base64.b64decode(secret_b64)
expected = hmac.new(secret, f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, v1)
import crypto from "node:crypto";
// SECRETS maps kid -> the base64 secret you captured at /hmac/keys/rotate.
function verifyWebhook(rawBody, signatureHeader, SECRETS) {
const parts = Object.fromEntries(
signatureHeader.split(",").map((kv) => kv.split("="))
); // { t, v1, kid }
const secret = SECRETS[parts.kid];
if (!secret) return false;
const expected = crypto
.createHmac("sha256", Buffer.from(secret, "base64"))
.update(`${parts.t}.${rawBody}`)
.digest("hex");
// constant-time compare
const a = Buffer.from(expected), b = Buffer.from(parts.v1);
return a.length === b.length && crypto.timingSafeEqual(a, b);
}
// Express-style receiver (capture the raw body!):
// app.post("/zr", express.raw({ type: "*/*" }), (req, res) => {
// const eventId = req.header("X-ZenRows-Event-Id"); // dedup on this
// const sig = req.header("X-Signature"); // absent => unsigned
// if (sig && !verifyWebhook(req.body.toString(), sig, SECRETS)) return res.sendStatus(401);
// if (alreadyHandled(eventId)) return res.sendStatus(200);
// handle(JSON.parse(req.body.toString()));
// res.sendStatus(200);
// });
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.Base64;
public class VerifyWebhook {
static boolean verify(String rawBody, String t, String v1, String secretB64) throws Exception {
byte[] secret = Base64.getDecoder().decode(secretB64);
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(secret, "HmacSHA256"));
byte[] digest = mac.doFinal((t + "." + rawBody).getBytes(StandardCharsets.UTF_8));
StringBuilder hex = new StringBuilder();
for (byte x : digest) hex.append(String.format("%02x", x));
return MessageDigest.isEqual(
hex.toString().getBytes(StandardCharsets.UTF_8), v1.getBytes(StandardCharsets.UTF_8));
}
}
<?php
function verify($rawBody, $t, $v1, $secretB64) {
$secret = base64_decode($secretB64);
$expected = hash_hmac('sha256', $t . '.' . $rawBody, $secret);
return hash_equals($expected, $v1);
}
?>
package main
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
)
func verify(rawBody, t, v1, secretB64 string) bool {
secret, _ := base64.StdEncoding.DecodeString(secretB64)
mac := hmac.New(sha256.New, secret)
mac.Write([]byte(t + "." + rawBody))
expected := hex.EncodeToString(mac.Sum(nil))
return hmac.Equal([]byte(expected), []byte(v1))
}
require 'openssl'
require 'base64'
def verify(raw_body, t, v1, secret_b64)
secret = Base64.decode64(secret_b64)
expected = OpenSSL::HMAC.hexdigest('SHA256', secret, "#{t}.#{raw_body}")
OpenSSL.fixed_length_secure_compare(expected, v1)
rescue StandardError
expected == v1
end
# Verification happens in your receiver, not in a request. With openssl:
# T and BODY come from the delivery; SECRET_B64 is your base64 key.
key_hex=$(printf '%s' "$SECRET_B64" | base64 -d | xxd -p -c 256)
printf '%s' "${T}.${BODY}" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$key_hex"
# Compare the printed hex digest against the v1 value from X-Signature.
5. Reruns and retrying failures
POST /jobs/{job_id}/rerun replays a job as a new run. Add ?status=failed to re-run only the failures (append ,pending to also include tasks that never started).
Already-successful tasks carry over, so you only re-scrape, and only pay for, what failed. The response reports retried_tasks and inherited_tasks.
## pip install requests
import requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
# Full rerun:
requests.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun', headers=headers)
# Retry only the failures:
response = requests.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun',
headers=headers, params={'status': 'failed'})
data = response.json()
print('new run', data['latest_run']['run_id'], 'retried', data['retried_tasks'], 'inherited', data['inherited_tasks'])
// Full rerun: replay every task as a new run:
const full = await api("POST", "/jobs/JOB_ID/rerun");
// Retry only failures (add ",pending" to also re-run tasks that never started):
const retry = await api("POST", "/jobs/JOB_ID/rerun", { query: { status: "failed" } });
console.log("new run", retry.latest_run.run_id, "retried", retry.retried_tasks, "inherited", retry.inherited_tasks);
import org.apache.hc.client5.http.fluent.Request;
public class Rerun {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
// Full rerun:
Request.post("https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun")
.addHeader("X-API-Key", key).execute().discardContent();
// Retry only the failures:
String response = Request.post("https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun?status=failed")
.addHeader("X-API-Key", key).execute().returnContent().asString();
System.out.println(response); // includes retried_tasks and inherited_tasks
}
}
<?php
$key = 'YOUR_ZENROWS_API_KEY';
// Retry only the failures (drop the query string for a full rerun):
$ch = curl_init('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun?status=failed');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key"]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL; // includes retried_tasks and inherited_tasks
curl_close($ch);
?>
package main
import (
"io"
"log"
"net/http"
)
func main() {
// Retry only the failures (drop the query string for a full rerun).
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun?status=failed", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body)) // includes retried_tasks and inherited_tasks
}
# gem install faraday
require 'faraday'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
# Full rerun:
conn.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun')
# Retry only the failures:
res = conn.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun') do |req|
req.params['status'] = 'failed'
end
print(res.body) # includes retried_tasks and inherited_tasks
# Full rerun:
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
# Retry only the failures:
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun?status=failed" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
6. Cost
Estimate spend before you submit, and read actual spend after a run.Estimate cost
Pricing is per successful request by tier, and the rate card is small and stable, so you can estimate cost on the client with no API call.- Basic request (base)
- 5x multiplier (JavaScript rendering)
- 10x multiplier (premium proxy)
- 25x multiplier (both)
- a 1x to 25x range for
mode=auto
mode=auto.
The estimate assumes each URL succeeds once, so it is a single number unless a task uses
mode=auto. For more details on multipliers, see the Pricing Page.def task_cost(params):
if str(params.get('mode')) == 'auto':
return (1, 25)
js = str(params.get('js_render')) == 'true'
proxy = str(params.get('premium_proxy')) == 'true'
c = 25 if js and proxy else 10 if proxy else 5 if js else 1
return (c, c)
def estimate(body):
job = body.get('zenrows_params', {})
lo = hi = 0
for t in body.get('tasks', []):
a, b = task_cost({**job, **t.get('zenrows_params', {})})
lo += a
hi += b
return {'tasks': len(body.get('tasks', [])), 'min': lo, 'max': hi, 'exact': lo == hi}
const TIERS = { base: 1, js: 5, proxy: 10, both: 25 }; // usage per successful request
function taskCost(params = {}) {
const v = (x) => String(x) === "true";
if (String(params.mode) === "auto") return { min: 1, max: 25 }; // dynamic
const js = v(params.js_render), proxy = v(params.premium_proxy);
const c = js && proxy ? TIERS.both : proxy ? TIERS.proxy : js ? TIERS.js : TIERS.base;
return { min: c, max: c };
}
function estimate(submitBody) {
const job = submitBody.zenrows_params ?? {};
let min = 0, max = 0;
for (const t of submitBody.tasks ?? []) {
const { min: a, max: b } = taskCost({ ...job, ...(t.zenrows_params ?? {}) });
min += a; max += b;
}
return { tasks: (submitBody.tasks ?? []).length, min, max, exact: min === max };
}
import java.util.List;
import java.util.Map;
public class Estimate {
static int[] taskCost(Map<String, String> p) {
if ("auto".equals(p.get("mode"))) return new int[]{1, 25};
boolean js = "true".equals(p.get("js_render"));
boolean proxy = "true".equals(p.get("premium_proxy"));
int c = js && proxy ? 25 : proxy ? 10 : js ? 5 : 1;
return new int[]{c, c};
}
// Merge job-level and per-task params before calling taskCost for each task,
// then sum the min and max across all tasks.
static int[] estimate(List<Map<String, String>> mergedTaskParams) {
int min = 0, max = 0;
for (Map<String, String> p : mergedTaskParams) {
int[] c = taskCost(p);
min += c[0];
max += c[1];
}
return new int[]{min, max};
}
}
<?php
function task_cost($params) {
if (($params['mode'] ?? '') === 'auto') return [1, 25];
$js = ($params['js_render'] ?? '') === 'true';
$proxy = ($params['premium_proxy'] ?? '') === 'true';
$c = $js && $proxy ? 25 : ($proxy ? 10 : ($js ? 5 : 1));
return [$c, $c];
}
function estimate($body) {
$job = $body['zenrows_params'] ?? [];
$min = 0; $max = 0;
foreach (($body['tasks'] ?? []) as $t) {
[$a, $b] = task_cost(array_merge($job, $t['zenrows_params'] ?? []));
$min += $a; $max += $b;
}
return ['tasks' => count($body['tasks'] ?? []), 'min' => $min, 'max' => $max, 'exact' => $min === $max];
}
?>
package main
func taskCost(p map[string]string) (int, int) {
if p["mode"] == "auto" {
return 1, 25
}
js := p["js_render"] == "true"
proxy := p["premium_proxy"] == "true"
switch {
case js && proxy:
return 25, 25
case proxy:
return 10, 10
case js:
return 5, 5
default:
return 1, 1
}
}
// Merge job-level and per-task params before calling taskCost for each task,
// then sum the min and max across all tasks.
func estimate(mergedTaskParams []map[string]string) (int, int) {
min, max := 0, 0
for _, p := range mergedTaskParams {
lo, hi := taskCost(p)
min += lo
max += hi
}
return min, max
}
def task_cost(params)
return [1, 25] if params['mode'].to_s == 'auto'
js = params['js_render'].to_s == 'true'
proxy = params['premium_proxy'].to_s == 'true'
c = js && proxy ? 25 : proxy ? 10 : js ? 5 : 1
[c, c]
end
def estimate(body)
job = body['zenrows_params'] || {}
min = 0; max = 0
(body['tasks'] || []).each do |t|
a, b = task_cost(job.merge(t['zenrows_params'] || {}))
min += a; max += b
end
{ tasks: (body['tasks'] || []).length, min: min, max: max, exact: min == max }
end
# Cost estimation is client-side arithmetic on the rate card, not an API call.
# Tiers (usage per successful request): base=1, js_render=5, premium_proxy=10, both=25, mode=auto=1..25.
# Sum the tier for each task. Example with jq over a submit body in body.json:
jq '[.tasks[] | (.zenrows_params // {}) as $p
| (if ($p.mode? == "auto") then 25
elif ($p.js_render? == "true" and $p.premium_proxy? == "true") then 25
elif ($p.premium_proxy? == "true") then 10
elif ($p.js_render? == "true") then 5
else 1 end)] | add' body.json
Check actual cost
Once a run finishes, you can read the usage it actually consumed at two levels:- Run total: the
spendobject onlatest_run.stats, read from theGET /jobs/{id}response shown below. - Per task: the
spendobject on each result row, wherespend.total.creditsis the cost across all attempts andspend.last_attempt.creditsis the cost of the most recent attempt. Read these while paging results, as shown in Download content.
## pip install requests
import requests
job = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'}).json()
spend = job['latest_run']['stats'].get('spend')
if spend:
print('usage:', spend['credits'], 'cost:', spend['cost'])
const job = await api("GET", "/jobs/JOB_ID");
const spend = job.latest_run.stats.spend; // { credits, cost }, absent until there's spend
if (spend) console.log("usage:", spend.credits, "cost:", spend.cost);
import org.apache.hc.client5.http.fluent.Request;
public class ActualCost {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response); // read latest_run.stats.spend from this JSON
}
}
<?php
$ch = curl_init('https://async.api.zenrows.com/v1/jobs/JOB_ID');
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$job = json_decode(curl_exec($ch), true);
curl_close($ch);
$spend = $job['latest_run']['stats']['spend'] ?? null;
if ($spend) echo "usage: {$spend['credits']} cost: {$spend['cost']}" . PHP_EOL;
?>
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body)) // read latest_run.stats.spend from this JSON
}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
job = JSON.parse(conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID') { |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
}.body)
spend = job.dig('latest_run', 'stats', 'spend')
puts "usage: #{spend['credits']} cost: #{spend['cost']}" if spend
# Read latest_run.stats.spend from the job response:
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
7. Reference
Behaviors and error details that apply across the API.Idempotency
Send anIdempotency-Key header on a submit to guard against accidentally creating duplicate jobs. Each key can be used once: the first request with a given key creates the job, and any later request that reuses the same key returns 409 idempotency_key_conflict, whether or not the body is identical. The API does not store or replay the original response.
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json',
'Idempotency-Key': 'nightly-2026-06-08'}
payload = {'type': 'regular', 'status': 'closed',
'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/'}]}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.status_code, response.json())
const job = await api("POST", "/jobs", {
headers: { "Idempotency-Key": "nightly-2026-06-08" },
body: { type: "regular", status: "closed", tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/" }] },
});
console.log(job.job_id);
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class Idempotent {
public static void main(final String... args) throws Exception {
String body = "{\"type\":\"regular\",\"status\":\"closed\",\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/\"}]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.addHeader("Idempotency-Key", "nightly-2026-06-08")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
<?php
$ch = curl_init('https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode([
'type' => 'regular', 'status' => 'closed',
'tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/']],
]));
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'X-API-Key: YOUR_ZENROWS_API_KEY',
'Idempotency-Key: nightly-2026-06-08',
'Content-Type: application/json',
]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{"type":"regular","status":"closed","tasks":[{"url":"https://www.scrapingcourse.com/ecommerce/"}]}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Idempotency-Key", "nightly-2026-06-08")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(resp.StatusCode, string(body))
}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Idempotency-Key'] = 'nightly-2026-06-08'
req.headers['Content-Type'] = 'application/json'
req.body = { type: 'regular', status: 'closed',
tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/' }] }.to_json
end
print(res.status)
print(res.body)
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" \
-H "Idempotency-Key: nightly-2026-06-08" \
-H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "closed", "tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/" }] }'
An idempotency key is not a safe-retry token, because reusing it always returns
409. If a submit fails midway, such as a dropped connection or a 503, do not resend with the same key. First check whether the job was created by listing your recent jobs, then either use its job_id or submit again with a new key. A late 503 can leave a visible job with a stopped run, which is safe to DELETE.Error handling reference
Errors use RFC 7807 problem JSON with a stablecode you can branch on. Task-level failures are not HTTP errors; they appear as failed rows in the results, each with its own error object. A successful POST /jobs returns 201 Created (synchronous) or 202 Accepted (asynchronous, for 10,000+ tasks or a CSV input); both carry the same response body.
| HTTP | code | Meaning |
|---|---|---|
| 400 | invalid_argument | Validation failed; invalid_tasks[] lists {index, reason}. |
| 401 | unauthenticated | Missing or invalid API key. |
| 402 | payment_required | No usage available. |
| 404 | not_found | Job, run, or task missing or not owned by you. |
| 409 | conflict / run_not_terminal / idempotency_key_conflict | State conflict; see detail. |
| 422 | (on /content) | Task failed; the body is the stored error. |
| 429 | quota_exceeded | Account limit reached, such as the maximum of 3 concurrent active jobs. |
| 503 | internal | Transient; safe to retry with a backoff. |
code and, for a 400, the invalid_tasks array from the problem body.
## pip install requests
import requests
r = requests.post('https://async.api.zenrows.com/v1/jobs',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'},
json={'type': 'regular', 'status': 'closed', 'tasks': [{'url': 'not-a-url'}]})
if not r.ok:
problem = r.json() # RFC 7807: { type, title, status, detail, code, invalid_tasks? }
print(r.status_code, problem.get('code'), problem.get('invalid_tasks'))
// The api() helper throws an object carrying the structured problem details.
try {
await api("POST", "/jobs", { body: { type: "regular", status: "closed", tasks: [{ url: "not-a-url" }] } });
} catch (e) {
console.error(e.status, e.code, e.problem?.invalid_tasks);
}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ClassicHttpResponse;
import org.apache.hc.core5.http.ContentType;
import org.apache.hc.core5.http.io.entity.EntityUtils;
public class ErrorHandling {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String body = "{\"type\":\"regular\",\"status\":\"closed\",\"tasks\":[{\"url\":\"not-a-url\"}]}";
ClassicHttpResponse resp = (ClassicHttpResponse) Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnResponse();
if (resp.getCode() >= 400) {
JsonNode problem = M.readTree(EntityUtils.toString(resp.getEntity())); // RFC 7807
System.out.println(resp.getCode() + " " + problem.path("code").asText() + " " + problem.path("invalid_tasks"));
}
}
}
<?php
$ch = curl_init('https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode(['type' => 'regular', 'status' => 'closed', 'tasks' => [['url' => 'not-a-url']]]));
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if ($status >= 400) {
$problem = json_decode($res, true); // RFC 7807
echo $status, ' ', $problem['code'] ?? '', PHP_EOL;
print_r($problem['invalid_tasks'] ?? []);
}
?>
package main
import (
"bytes"
"encoding/json"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{"type":"regular","status":"closed","tasks":[{"url":"not-a-url"}]}`)
req, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode >= 400 {
var problem struct {
Code string `json:"code"`
InvalidTasks []any `json:"invalid_tasks"`
}
json.Unmarshal(body, &problem) // RFC 7807
log.Println(resp.StatusCode, problem.Code, problem.InvalidTasks)
}
}
# gem install faraday
require 'faraday'
require 'json'
res = Faraday.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = { type: 'regular', status: 'closed', tasks: [{ url: 'not-a-url' }] }.to_json
end
unless res.success?
problem = JSON.parse(res.body) # RFC 7807
puts "#{res.status} #{problem['code']}"
p problem['invalid_tasks']
end
# -i shows the status line; the body is the RFC 7807 problem with code and invalid_tasks.
curl -i -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "closed", "tasks": [{ "url": "not-a-url" }] }'
Troubleshooting
Common issues you may hit while integrating. See the full Troubleshooting page for the complete list.401 unauthenticated: send your key in theX-API-Keyheader (never in the URL), and confirm your account has private-beta access.400 invalid_argument: the body failed validation. Read theinvalid_tasksarray to see which URL failed and why.429 quota_exceeded: you have 3 jobs running already. Wait for one to finish, since 3 concurrent jobs is the limit.503when opening a job: open (queue-mode) jobs are still being finalized during the beta. Use a closed job or a CSV upload for now.- Download link returns an error: each
result_urlis valid for 2 hours. Re-list the results to get a fresh link rather than reusing a stored one. - Webhook never arrives: use an HTTPS receiver, test it with
POST /webhook/test, and deduplicate on theX-ZenRows-Event-Idheader, since deliveries are at-least-once.
FAQ (Frequently Asked Questions)
Quick answers for developers. See the full FAQ for more.What's the base URL, and do I need a separate key?
What's the base URL, and do I need a separate key?
All requests go to
https://async.api.zenrows.com/v1, authenticated with the X-API-Key header. The Batch Scraper API uses the same API key as the Universal Scraper API.Which languages are supported?
Which languages are supported?
Any language with an HTTP client. The examples in this guide cover Python, Node.js, Java, PHP, Go, Ruby, and cURL, and the same REST calls work anywhere.
How many URLs and jobs can I run?
How many URLs and jobs can I run?
Up to 100,000 URLs per job submit, 10,000 per add-tasks call on an open job, and 3 active jobs concurrently. CSV uploads support 50 MB / 100,000 rows.
Is the idempotency key safe to retry with?
Is the idempotency key safe to retry with?
No. Each key can be used once, and reusing it always returns
409 idempotency_key_conflict. If a submit fails midway, check whether the job was created before resending with a new key.How am I billed?
How am I billed?
In usage, per successful request, at the same rates as the Universal Scraper API. Failed requests are never charged.