> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Developer Guide: REST API
> Drive the ZenRows Batch Scraper API directly over HTTP. Each request is shown in Python, Node.js, Java, PHP, Go, Ruby, and cURL, with the full original logic preserved.
The Batch Scraper API is in **private beta**. You need an invited account and an active ZenRows API key. Replace `YOUR_ZENROWS_API_KEY` in every example with your key.
## Basics
* **Base URL:** `https://async.api.zenrows.com/v1`
* **Authentication:** send `X-API-Key: YOUR_ZENROWS_API_KEY` on every request. The key travels in a header, never in the URL.
* **Bodies:** send JSON with `Content-Type: application/json` and receive JSON. Errors come back as `application/problem+json`, a standard JSON error format (RFC 7807) with a stable `code` you can branch on.
The Node.js examples below reuse this small wrapper, which adds authentication, sets the `Content-Type` header when there is a body, and turns error responses into thrown objects:
```javascript Node.js theme={null}
const BASE = process.env.BATCH_API_URL ?? "https://async.api.zenrows.com/v1";
const API_KEY = process.env.ZENROWS_API_KEY;
async function api(method, path, { body, query, headers } = {}) {
const url = new URL(BASE + path);
for (const [k, v] of Object.entries(query ?? {})) {
if (v !== undefined && v !== null) url.searchParams.set(k, v);
}
const res = await fetch(url, {
method,
headers: {
"X-API-Key": API_KEY,
...(body ? { "Content-Type": "application/json" } : {}),
...headers,
},
body: body ? JSON.stringify(body) : undefined,
});
if (res.status === 204) return null;
const isJson = (res.headers.get("content-type") ?? "").includes("json");
const payload = isJson ? await res.json() : await res.arrayBuffer();
if (!res.ok) {
// RFC 7807 problem: { type, title, status, detail, code, invalid_tasks? }
const err = new Error(payload?.detail || payload?.title || `HTTP ${res.status}`);
err.status = res.status;
err.code = payload?.code;
err.problem = payload;
throw err;
}
return payload;
}
```
## 1. Create your jobs
There are several ways to create a job. Pick the one that matches how your URLs arrive.
### Submit a job with all URLs up front
Create a **closed job** (one where you know all URLs up front) with `POST /jobs`. The response includes a `job_id` and the initial run status.
`POST /jobs` returns **`201 Created`** for submissions under 10,000 inline tasks and **`202 Accepted`** for 10,000 or more tasks (or a CSV input). The request shape and response body are identical, so treat both the same: store `job_id`, then poll until `stats.completed >= stats.total` (or use a webhook). On a `202`, `stats.total` is correct immediately, but `GET /jobs/{id}/results` may return partial pages for a few seconds while the task rows finish writing.
```python Python theme={null}
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
payload = {
'type': 'regular',
'status': 'closed',
'zenrows_params': {'js_render': 'true', 'premium_proxy': 'true'},
'tasks': [
{'url': 'https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie', 'external_id': 'order-1'},
{'url': 'https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket', 'external_id': 'order-2'},
],
}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.json())
```
```javascript Node.js theme={null}
// Reuses the api() helper from the Basics section.
const submitted = await api("POST", "/jobs", {
body: {
type: "regular",
status: "closed",
zenrows_params: { js_render: "true", premium_proxy: "true" },
tasks: [
{ url: "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie", external_id: "order-1" },
{ url: "https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket", external_id: "order-2" },
],
},
});
console.log("submitted", submitted.job_id, submitted.latest_run.status);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class SubmitJob {
public static void main(final String... args) throws Exception {
String body = "{"
+ "\"type\":\"regular\",\"status\":\"closed\","
+ "\"zenrows_params\":{\"js_render\":\"true\",\"premium_proxy\":\"true\"},"
+ "\"tasks\":["
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie\",\"external_id\":\"order-1\"},"
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket\",\"external_id\":\"order-2\"}"
+ "]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
'regular',
'status' => 'closed',
'zenrows_params' => ['js_render' => 'true', 'premium_proxy' => 'true'],
'tasks' => [
['url' => 'https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie', 'external_id' => 'order-1'],
['url' => 'https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket', 'external_id' => 'order-2'],
],
]);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
echo $response . PHP_EOL;
curl_close($ch);
?>
```
```go Go theme={null}
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "premium_proxy": "true" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie", "external_id": "order-1" },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket", "external_id": "order-2" }
]
}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
payload = {
type: 'regular',
status: 'closed',
zenrows_params: { js_render: 'true', premium_proxy: 'true' },
tasks: [
{ url: 'https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie', external_id: 'order-1' },
{ url: 'https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket', external_id: 'order-2' }
]
}
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = payload.to_json
end
print(res.body)
```
```bash cURL theme={null}
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "premium_proxy": "true" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie", "external_id": "order-1" },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket", "external_id": "order-2" }
]
}'
```
### Configure the scrape and per-URL options
`zenrows_params` applies to every URL in the job. A per-task `zenrows_params` overrides the job-level keys for that URL, so you can set the common case once and tune exceptions such as a different `proxy_country`.
Common keys:
* [`js_render`](/universal-scraper-api/features/js-rendering)
* [`premium_proxy`](/universal-scraper-api/features/premium-proxy)
* [`proxy_country`](/universal-scraper-api/features/proxy-country)
* [`js_instructions`](/universal-scraper-api/features/js-instructions)
* [`wait_for`](/universal-scraper-api/features/wait-for)
* [`wait`](/universal-scraper-api/features/wait)
* [`block_resources`](/universal-scraper-api/features/block-resources)
* [`custom_headers`](/universal-scraper-api/features/headers)
* [`json_response`](/universal-scraper-api/features/json-response)
* [`css_extractor`](/universal-scraper-api/features/css-extractor)
* [`autoparse`](/universal-scraper-api/features/autoparse)
* `response_type` ([`markdown`](/universal-scraper-api/features/markdown), [`plaintext`](/universal-scraper-api/features/plaintext), or [`pdf`](/universal-scraper-api/features/pdf))
* [`outputs`](/universal-scraper-api/features/output-filters)
* [`original_status`](/universal-scraper-api/features/other#original-http-code)
* [`allowed_status_codes`](/universal-scraper-api/features/other#return-content-on-error)
* [`session_id`](/universal-scraper-api/features/other#session-id)
```python Python theme={null}
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
payload = {
'type': 'regular',
'status': 'closed',
'zenrows_params': {'js_render': 'true', 'proxy_country': 'us', 'response_type': 'markdown'},
'tasks': [
{'url': 'https://www.scrapingcourse.com/ecommerce/product/artemis-running-short',
'external_id': 'order-42', 'zenrows_params': {'proxy_country': 'de'}},
{'url': 'https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee',
'metadata': {'sku': 'ABC-1'}},
],
}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.json())
```
```javascript Node.js theme={null}
const job = await api("POST", "/jobs", {
body: {
type: "regular",
status: "closed",
zenrows_params: { js_render: "true", proxy_country: "us", response_type: "markdown" },
tasks: [
{ url: "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short",
external_id: "order-42", zenrows_params: { proxy_country: "de" } },
{ url: "https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee",
metadata: { sku: "ABC-1" } },
],
},
});
console.log(job.job_id);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class ConfigureJob {
public static void main(final String... args) throws Exception {
String body = "{"
+ "\"type\":\"regular\",\"status\":\"closed\","
+ "\"zenrows_params\":{\"js_render\":\"true\",\"proxy_country\":\"us\",\"response_type\":\"markdown\"},"
+ "\"tasks\":["
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/artemis-running-short\",\"external_id\":\"order-42\",\"zenrows_params\":{\"proxy_country\":\"de\"}},"
+ "{\"url\":\"https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee\",\"metadata\":{\"sku\":\"ABC-1\"}}"
+ "]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
'regular',
'status' => 'closed',
'zenrows_params' => ['js_render' => 'true', 'proxy_country' => 'us', 'response_type' => 'markdown'],
'tasks' => [
['url' => 'https://www.scrapingcourse.com/ecommerce/product/artemis-running-short',
'external_id' => 'order-42', 'zenrows_params' => ['proxy_country' => 'de']],
['url' => 'https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee',
'metadata' => ['sku' => 'ABC-1']],
],
]);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
```
```go Go theme={null}
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "proxy_country": "us", "response_type": "markdown" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short", "external_id": "order-42", "zenrows_params": { "proxy_country": "de" } },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee", "metadata": { "sku": "ABC-1" } }
]
}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
payload = {
type: 'regular',
status: 'closed',
zenrows_params: { js_render: 'true', proxy_country: 'us', response_type: 'markdown' },
tasks: [
{ url: 'https://www.scrapingcourse.com/ecommerce/product/artemis-running-short',
external_id: 'order-42', zenrows_params: { proxy_country: 'de' } },
{ url: 'https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee',
metadata: { sku: 'ABC-1' } }
]
}
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = payload.to_json
end
print(res.body)
```
```bash cURL theme={null}
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"type": "regular",
"status": "closed",
"zenrows_params": { "js_render": "true", "proxy_country": "us", "response_type": "markdown" },
"tasks": [
{ "url": "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short", "external_id": "order-42", "zenrows_params": { "proxy_country": "de" } },
{ "url": "https://www.scrapingcourse.com/ecommerce/product/atomic-endurance-tee", "metadata": { "sku": "ABC-1" } }
]
}'
```
The API returns your `external_id` unchanged on every result, so you can match results back to your own records without tracking the API's task IDs. It does not need to be unique.
### Open jobs: adding tasks in batches
Open (queue-mode) jobs are still being finalized during the private beta and may not be available yet. If `POST /jobs` with `status: "open"` returns a `503`, use a closed job or a CSV upload for now.
Create an open job, add tasks with `POST /jobs/{job_id}/tasks` over time, then close it by sending `last_batch: true` on the final batch. Each add-tasks call accepts up to 10,000 URLs.
```python Python theme={null}
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
job = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers,
data=json.dumps({'type': 'regular', 'status': 'open',
'zenrows_params': {'js_render': 'true'}})).json()
job_id = job['job_id']
requests.post(f'https://async.api.zenrows.com/v1/jobs/{job_id}/tasks', headers=headers,
data=json.dumps({'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/page/1'}]}))
requests.post(f'https://async.api.zenrows.com/v1/jobs/{job_id}/tasks', headers=headers,
data=json.dumps({'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/page/2'}], 'last_batch': True}))
```
```javascript Node.js theme={null}
const { job_id } = await api("POST", "/jobs", {
body: { type: "regular", status: "open", zenrows_params: { js_render: "true" } },
});
await api("POST", `/jobs/${job_id}/tasks`, {
body: { tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/page/1" }] },
});
await api("POST", `/jobs/${job_id}/tasks`, {
body: { tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/page/2" }], last_batch: true }, // closes the job
});
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class OpenJob {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String created = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", key)
.bodyString("{\"type\":\"regular\",\"status\":\"open\",\"zenrows_params\":{\"js_render\":\"true\"}}", ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(created); // read job_id from this response
String jobId = "JOB_ID";
Request.post("https://async.api.zenrows.com/v1/jobs/" + jobId + "/tasks")
.addHeader("X-API-Key", key)
.bodyString("{\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/page/1\"}]}", ContentType.APPLICATION_JSON)
.execute().discardContent();
Request.post("https://async.api.zenrows.com/v1/jobs/" + jobId + "/tasks")
.addHeader("X-API-Key", key)
.bodyString("{\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/page/2\"}],\"last_batch\":true}", ContentType.APPLICATION_JSON)
.execute().discardContent();
}
}
```
```php PHP theme={null}
'regular', 'status' => 'open', 'zenrows_params' => ['js_render' => 'true']]);
$jobId = $job['job_id'];
post("/jobs/$jobId/tasks", ['tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/page/1']]]);
post("/jobs/$jobId/tasks", ['tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/page/2']], 'last_batch' => true]);
?>
```
```go Go theme={null}
package main
import (
"bytes"
"log"
"net/http"
)
func post(path string, body string) {
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1"+path, bytes.NewBufferString(body))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
}
func main() {
// Create the open job, then read job_id from the response body.
post("/jobs", `{"type":"regular","status":"open","zenrows_params":{"js_render":"true"}}`)
jobId := "JOB_ID"
post("/jobs/"+jobId+"/tasks", `{"tasks":[{"url":"https://www.scrapingcourse.com/ecommerce/page/1"}]}`)
post("/jobs/"+jobId+"/tasks", `{"tasks":[{"url":"https://www.scrapingcourse.com/ecommerce/page/2"}],"last_batch":true}`)
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
conn.headers['Content-Type'] = 'application/json'
job = JSON.parse(conn.post('https://async.api.zenrows.com/v1/jobs',
{ type: 'regular', status: 'open', zenrows_params: { js_render: 'true' } }.to_json).body)
job_id = job['job_id']
conn.post("https://async.api.zenrows.com/v1/jobs/#{job_id}/tasks",
{ tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/page/1' }] }.to_json)
conn.post("https://async.api.zenrows.com/v1/jobs/#{job_id}/tasks",
{ tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/page/2' }], last_batch: true }.to_json)
```
```bash cURL theme={null}
# 1) Create the open job and note its job_id.
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "open", "zenrows_params": { "js_render": "true" } }'
# 2) Add a batch of tasks.
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/page/1" }] }'
# 3) Add the final batch and close the job.
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/page/2" }], "last_batch": true }'
```
`last_batch: true` (or `POST /jobs/{id}/close`) locks the job, and later adds return `409`. To abandon a run that is still in flight, call `POST /jobs/{id}/stop`. A very large submission also keeps ingesting task rows for a few seconds after it responds; `POST /jobs/{id}/tasks` can return `409` during that window, so retry shortly.
### Large inputs: CSV upload
For lists too large to inline, upload a CSV in three steps:
1. Create an input slot with `POST /job_inputs`.
2. PUT the file to the returned presigned URL using the exact headers it specifies
3. Submit a job referencing `file_input_id`.
The `fields` map points the canonical `url` (required) and `external_id` (optional) fields at your CSV columns by header name (with `header: true`) or 0-based index.
Limits: 50 MB / 100,000 rows. Upload slots live 24 hours.
```python Python theme={null}
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
# 1) Create the slot.
slot = requests.post('https://async.api.zenrows.com/v1/job_inputs',
headers={**headers, 'Content-Type': 'application/json'},
data=json.dumps({'type': 'csv',
'csv': {'header': True, 'fields': {'url': 'URL', 'external_id': 'Customer Ref'}}})).json()
# 2) Upload the file with the exact content-type the slot requires.
with open('urls.csv', 'rb') as f:
requests.put(slot['upload']['url'], headers=slot['upload']['headers'], data=f)
# 3) Submit referencing the slot.
job = requests.post('https://async.api.zenrows.com/v1/jobs',
headers={**headers, 'Content-Type': 'application/json'},
data=json.dumps({'type': 'regular', 'status': 'closed',
'file_input_id': slot['file_input_id'], 'zenrows_params': {'js_render': 'true'}})).json()
print(job['job_id'])
```
```javascript Node.js theme={null}
import { readFile } from "node:fs/promises";
// 1) Create the slot.
const slot = await api("POST", "/job_inputs", {
body: { type: "csv", csv: { header: true, fields: { url: "URL", external_id: "Customer Ref" } } },
});
// 2) Upload the body with the EXACT content-type the slot requires.
const csvBytes = await readFile("urls.csv");
const put = await fetch(slot.upload.url, {
method: slot.upload.method, // "PUT"
headers: slot.upload.headers, // e.g. { "Content-Type": "text/csv" }
body: csvBytes,
});
if (!put.ok) throw new Error(`upload failed: ${put.status}`);
// 3) Submit referencing the slot instead of inline tasks.
const job = await api("POST", "/jobs", {
body: { type: "regular", status: "closed", file_input_id: slot.file_input_id, zenrows_params: { js_render: "true" } },
});
console.log(job.job_id);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
import java.io.File;
public class CsvUpload {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
// 1) Create the slot, then read upload.url, upload.headers, and file_input_id from the response.
String slot = Request.post("https://async.api.zenrows.com/v1/job_inputs")
.addHeader("X-API-Key", key)
.bodyString("{\"type\":\"csv\",\"csv\":{\"header\":true,\"fields\":{\"url\":\"URL\",\"external_id\":\"Customer Ref\"}}}", ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(slot);
// 2) PUT the file to the presigned URL with the content-type the slot requires.
Request.put("PRESIGNED_UPLOAD_URL")
.bodyFile(new File("urls.csv"), ContentType.create("text/csv"))
.execute().discardContent();
// 3) Submit referencing the slot.
String job = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", key)
.bodyString("{\"type\":\"regular\",\"status\":\"closed\",\"file_input_id\":\"FILE_INPUT_ID\",\"zenrows_params\":{\"js_render\":\"true\"}}", ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(job);
}
}
```
```php PHP theme={null}
'csv',
'csv' => ['header' => true, 'fields' => ['url' => 'URL', 'external_id' => 'Customer Ref']],
]));
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$slot = json_decode(curl_exec($ch), true);
curl_close($ch);
// 2) Upload the file with the exact content-type the slot requires.
$ch = curl_init($slot['upload']['url']);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_POSTFIELDS, file_get_contents('urls.csv'));
curl_setopt($ch, CURLOPT_HTTPHEADER, $slot['upload']['headers']);
curl_exec($ch);
curl_close($ch);
// 3) Submit referencing the slot.
$ch = curl_init('https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode([
'type' => 'regular', 'status' => 'closed',
'file_input_id' => $slot['file_input_id'], 'zenrows_params' => ['js_render' => 'true'],
]));
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
```
```go Go theme={null}
package main
import (
"bytes"
"io"
"log"
"net/http"
"os"
)
func main() {
key := "YOUR_ZENROWS_API_KEY"
// 1) Create the slot, then read upload.url, upload.headers, and file_input_id from the response.
slotReq, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/job_inputs",
bytes.NewBufferString(`{"type":"csv","csv":{"header":true,"fields":{"url":"URL","external_id":"Customer Ref"}}}`))
slotReq.Header.Set("X-API-Key", key)
slotReq.Header.Set("Content-Type", "application/json")
slotResp, err := http.DefaultClient.Do(slotReq)
if err != nil {
log.Fatalln(err)
}
defer slotResp.Body.Close()
slotBody, _ := io.ReadAll(slotResp.Body)
log.Println(string(slotBody))
// 2) PUT the file to the presigned URL with the content-type the slot requires.
file, _ := os.Open("urls.csv")
defer file.Close()
putReq, _ := http.NewRequest("PUT", "PRESIGNED_UPLOAD_URL", file)
putReq.Header.Set("Content-Type", "text/csv")
if _, err := http.DefaultClient.Do(putReq); err != nil {
log.Fatalln(err)
}
// 3) Submit referencing the slot.
jobReq, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs",
bytes.NewBufferString(`{"type":"regular","status":"closed","file_input_id":"FILE_INPUT_ID","zenrows_params":{"js_render":"true"}}`))
jobReq.Header.Set("X-API-Key", key)
jobReq.Header.Set("Content-Type", "application/json")
jobResp, err := http.DefaultClient.Do(jobReq)
if err != nil {
log.Fatalln(err)
}
defer jobResp.Body.Close()
jobBody, _ := io.ReadAll(jobResp.Body)
log.Println(string(jobBody))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
conn = Faraday.new
# 1) Create the slot.
slot = JSON.parse(conn.post('https://async.api.zenrows.com/v1/job_inputs',
{ type: 'csv', csv: { header: true, fields: { url: 'URL', external_id: 'Customer Ref' } } }.to_json,
{ 'X-API-Key' => key, 'Content-Type' => 'application/json' }).body)
# 2) Upload the file with the exact headers the slot requires.
conn.put(slot['upload']['url'], File.read('urls.csv'), slot['upload']['headers'])
# 3) Submit referencing the slot.
job = JSON.parse(conn.post('https://async.api.zenrows.com/v1/jobs',
{ type: 'regular', status: 'closed', file_input_id: slot['file_input_id'], zenrows_params: { js_render: 'true' } }.to_json,
{ 'X-API-Key' => key, 'Content-Type' => 'application/json' }).body)
puts job['job_id']
```
```bash cURL theme={null}
# 1) Create the slot. Read upload.url, upload.headers, and file_input_id from the response.
curl -X POST "https://async.api.zenrows.com/v1/job_inputs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "csv", "csv": { "header": true, "fields": { "url": "URL", "external_id": "Customer Ref" } } }'
# 2) Upload the file to the presigned URL with the content-type it requires.
curl -X PUT "PRESIGNED_UPLOAD_URL" -H "Content-Type: text/csv" --data-binary @urls.csv
# 3) Submit referencing the slot.
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "closed", "file_input_id": "FILE_INPUT_ID", "zenrows_params": { "js_render": "true" } }'
```
### Scheduled jobs
A scheduled job stores its URLs as a template, and each fire creates a new run. Provide exactly one of `at`, `rate`, or `calendar` (plus a `timezone` for `at` and `calendar`). Times of day must be full hours. The request below schedules a job to run every 6 hours.
```python Python theme={null}
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
payload = {
'type': 'scheduled', 'status': 'closed',
'schedule': {'rate': {'every': 6, 'unit': 'hour'}},
'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/'}],
}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.json())
```
```javascript Node.js theme={null}
const job = await api("POST", "/jobs", {
body: {
type: "scheduled", status: "closed",
schedule: { rate: { every: 6, unit: "hour" } },
tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/" }],
},
});
console.log(job.job_id);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class ScheduledJob {
public static void main(final String... args) throws Exception {
String body = "{\"type\":\"scheduled\",\"status\":\"closed\","
+ "\"schedule\":{\"rate\":{\"every\":6,\"unit\":\"hour\"}},"
+ "\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/\"}]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
'scheduled', 'status' => 'closed',
'schedule' => ['rate' => ['every' => 6, 'unit' => 'hour']],
'tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/']],
]);
$ch = curl_init('https://async.api.zenrows.com/v1/jobs');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
```
```go Go theme={null}
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{
"type": "scheduled", "status": "closed",
"schedule": { "rate": { "every": 6, "unit": "hour" } },
"tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/" }]
}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
payload = {
type: 'scheduled', status: 'closed',
schedule: { rate: { every: 6, unit: 'hour' } },
tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/' }]
}
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = payload.to_json
end
print(res.body)
```
```bash cURL theme={null}
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{
"type": "scheduled", "status": "closed",
"schedule": { "rate": { "every": 6, "unit": "hour" } },
"tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/" }]
}'
```
For a calendar schedule, replace the `schedule` object. The example below runs at 09:00 and 18:00 on Mon/Wed/Fri in Berlin time. Use an `at` object instead for a one-shot run at a single wall-clock time.
```json theme={null}
{
"schedule": {
"calendar": {
"times_of_day": ["09:00", "18:00"],
"cadence": { "weekly": { "days": ["mon", "wed", "fri"] } }
},
"timezone": "Europe/Berlin"
}
}
```
Manage a schedule with `PUT /jobs/{id}/schedule` to replace it, and `POST /jobs/{id}/schedule/state` with `{"schedule_state": "paused"}` or `{"schedule_state": "active"}` to pause or resume.
## 2. Track your jobs
Check progress and find your jobs once they are running.
### Check job status
Read a job with `GET /jobs/{job_id}`. Progress lives in `latest_run.stats`, where `completed` equals `successful` plus `failed`.
```python Python theme={null}
## pip install requests
import requests
response = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'})
job = response.json()
print(job['latest_run']['status'], job['latest_run']['stats'])
```
```javascript Node.js theme={null}
const job = await api("GET", "/jobs/JOB_ID");
console.log(job.latest_run.status, job.latest_run.stats);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class GetJob {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
print(res.body)
```
```bash cURL theme={null}
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
To wait for a run to finish, poll this endpoint until `latest_run.status` reaches a **terminal** state (`completed`, `stopped`, or `deleted`), backing off between calls rather than running a tight loop.
```python Python theme={null}
## pip install requests
import time, requests
def wait_for_job(job_id, timeout=600):
terminal = {'completed', 'stopped', 'deleted'}
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
deadline = time.time() + timeout
delay = 2.0
while True:
job = requests.get(f'https://async.api.zenrows.com/v1/jobs/{job_id}', headers=headers).json()
run = job.get('latest_run')
if run and run['status'] in terminal:
return job
if time.time() > deadline:
raise TimeoutError('timeout waiting for job')
time.sleep(delay)
delay = min(delay * 1.5, 15.0) # back off gradually, capped at 15s
job = wait_for_job('JOB_ID')
s = job['latest_run']['stats'] # { total, completed, successful, failed }
print(job['latest_run']['status'], f"{s['successful']}/{s['total']} successful")
```
```javascript Node.js theme={null}
async function waitForJob(jobId, { timeoutMs = 600_000 } = {}) {
const TERMINAL = new Set(["completed", "stopped", "deleted"]);
const deadline = Date.now() + timeoutMs;
let delay = 2000;
for (;;) {
const job = await api("GET", `/jobs/${jobId}`);
const run = job.latest_run;
if (run && TERMINAL.has(run.status)) return job;
if (Date.now() > deadline) throw new Error("timeout waiting for job");
await new Promise((r) => setTimeout(r, delay));
delay = Math.min(delay * 1.5, 15_000); // back off gradually, capped at 15s
}
}
const job = await waitForJob("JOB_ID");
const s = job.latest_run.stats; // { total, completed, successful, failed }
console.log(`${job.latest_run.status}: ${s.successful}/${s.total} successful`);
```
```java Java theme={null}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import java.util.Set;
public class WaitForJob {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
Set terminal = Set.of("completed", "stopped", "deleted");
long deadline = System.currentTimeMillis() + 600_000;
long delay = 2000;
while (true) {
String body = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
JsonNode run = M.readTree(body).path("latest_run");
if (terminal.contains(run.path("status").asText())) {
System.out.println(run.path("stats"));
break;
}
if (System.currentTimeMillis() > deadline) throw new RuntimeException("timeout waiting for job");
Thread.sleep(delay);
delay = Math.min((long) (delay * 1.5), 15_000); // back off gradually, capped at 15s
}
}
}
```
```php PHP theme={null}
$deadline) throw new RuntimeException('timeout waiting for job');
usleep((int)($delay * 1000000));
$delay = min($delay * 1.5, 15.0); // back off gradually, capped at 15s
}
?>
```
```go Go theme={null}
package main
import (
"encoding/json"
"io"
"log"
"net/http"
"time"
)
func getJob(id string) map[string]any {
req, _ := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/"+id, nil)
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var job map[string]any
json.Unmarshal(body, &job)
return job
}
func main() {
terminal := map[string]bool{"completed": true, "stopped": true, "deleted": true}
deadline := time.Now().Add(10 * time.Minute)
delay := 2 * time.Second
for {
job := getJob("JOB_ID")
run, _ := job["latest_run"].(map[string]any)
status, _ := run["status"].(string)
if run != nil && terminal[status] {
log.Println(run["stats"])
break
}
if time.Now().After(deadline) {
log.Fatalln("timeout waiting for job")
}
time.Sleep(delay)
if delay < 15*time.Second {
delay = time.Duration(float64(delay) * 1.5)
}
}
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
def get_job(id)
res = Faraday.get("https://async.api.zenrows.com/v1/jobs/#{id}") do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
JSON.parse(res.body)
end
terminal = ['completed', 'stopped', 'deleted']
deadline = Time.now + 600
delay = 2.0
loop do
job = get_job('JOB_ID')
run = job['latest_run']
if run && terminal.include?(run['status'])
p run['stats']
break
end
raise 'timeout waiting for job' if Time.now > deadline
sleep(delay)
delay = [delay * 1.5, 15.0].min # back off gradually, capped at 15s
end
```
```bash cURL theme={null}
# cURL has no loop of its own; poll with a small shell loop until the run is terminal.
delay=2
while :; do
status=$(curl -s "https://async.api.zenrows.com/v1/jobs/JOB_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" | jq -r '.latest_run.status')
echo "status: $status"
case "$status" in completed|stopped|deleted) break ;; esac
sleep "$delay"
delay=$(( delay < 15 ? delay + delay/2 + 1 : 15 )) # back off, capped at 15s
done
```
### List and paginate jobs
List your jobs with `GET /jobs`, optionally filtered by `status` and `type`, and page with `cursor` until `next_cursor` is null.
```python Python theme={null}
## pip install requests
import requests
response = requests.get('https://async.api.zenrows.com/v1/jobs',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'},
params={'status': 'closed', 'type': 'regular'})
print(response.json())
```
```javascript Node.js theme={null}
const page = await api("GET", "/jobs", { query: { status: "closed", type: "regular" } });
console.log(page.jobs);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class ListJobs {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs?status=closed&type=regular")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.params['status'] = 'closed'
req.params['type'] = 'regular'
end
print(res.body)
```
```bash cURL theme={null}
curl "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
The cursor loop is the same as for results, and `GET /jobs/{id}/runs` pages identically.
```python Python theme={null}
## pip install requests
import requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
cursor = None
while True:
page = requests.get('https://async.api.zenrows.com/v1/jobs', headers=headers,
params={'status': 'closed', 'type': 'regular', 'cursor': cursor}).json()
for job in page['jobs']:
latest = job.get('latest_run')
print(job['job_id'], job['status'], latest['stats']['total'] if latest else 0)
cursor = page['next_cursor']
if not cursor:
break
```
```javascript Node.js theme={null}
async function* iterJobs({ status, type } = {}) {
let cursor;
do {
const page = await api("GET", "/jobs", { query: { status, type, cursor } });
yield* page.jobs;
cursor = page.next_cursor;
} while (cursor);
}
for await (const job of iterJobs({ status: "closed", type: "regular" })) {
console.log(job.job_id, job.status, job.latest_run?.stats.total ?? 0);
}
```
```java Java theme={null}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
public class ListJobsPaged {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String cursor = null;
do {
String url = "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular"
+ (cursor != null ? "&cursor=" + cursor : "");
JsonNode page = M.readTree(Request.get(url).addHeader("X-API-Key", key)
.execute().returnContent().asString());
for (JsonNode job : page.path("jobs")) {
long total = job.path("latest_run").path("stats").path("total").asLong(0);
System.out.println(job.path("job_id").asText() + " " + job.path("status").asText() + " " + total);
}
cursor = page.path("next_cursor").asText(null);
} while (cursor != null && !cursor.isEmpty());
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"encoding/json"
"io"
"log"
"net/http"
)
func main() {
key := "YOUR_ZENROWS_API_KEY"
cursor := ""
for {
url := "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular"
if cursor != "" {
url += "&cursor=" + cursor
}
req, _ := http.NewRequest("GET", url, nil)
req.Header.Set("X-API-Key", key)
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var page struct {
Jobs []struct {
JobID string `json:"job_id"`
Status string `json:"status"`
LatestRun *struct {
Stats struct {
Total int `json:"total"`
} `json:"stats"`
} `json:"latest_run"`
} `json:"jobs"`
NextCursor string `json:"next_cursor"`
}
json.Unmarshal(body, &page)
for _, job := range page.Jobs {
total := 0
if job.LatestRun != nil {
total = job.LatestRun.Stats.Total
}
log.Println(job.JobID, job.Status, total)
}
cursor = page.NextCursor
if cursor == "" {
break
}
}
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
cursor = nil
loop do
res = Faraday.get('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = key
req.params['status'] = 'closed'
req.params['type'] = 'regular'
req.params['cursor'] = cursor if cursor
end
page = JSON.parse(res.body)
page['jobs'].each do |job|
total = job.dig('latest_run', 'stats', 'total') || 0
puts "#{job['job_id']} #{job['status']} #{total}"
end
cursor = page['next_cursor']
break unless cursor
end
```
```bash cURL theme={null}
cursor=""
while :; do
page=$(curl -s "https://async.api.zenrows.com/v1/jobs?status=closed&type=regular&cursor=$cursor" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY")
echo "$page" | jq -r '.jobs[] | [.job_id, .status, (.latest_run.stats.total // 0)] | @tsv'
cursor=$(echo "$page" | jq -r '.next_cursor // empty')
[ -z "$cursor" ] && break
done
```
## 3. Collect your results
Retrieve the scraped content once a run completes.
### Download content
List a run's results with `GET /jobs/{job_id}/results`, filtered by `status` (`successful`, `failed`, or `all`) and paged with the `cursor` query parameter until `next_cursor` is null.
Each row's `result_url` is a presigned link valid for 2 hours, so fetch it promptly.
```python Python theme={null}
## pip install requests
import requests
response = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'},
params={'status': 'successful'})
page = response.json()
for row in page['results']:
print(row.get('external_id') or row['task_id'], '->', row['result_url'])
print('next_cursor:', page['next_cursor'])
```
```javascript Node.js theme={null}
const page = await api("GET", "/jobs/JOB_ID/results", { query: { status: "successful" } });
for (const row of page.results) {
console.log(row.external_id ?? row.task_id, "->", row.result_url);
}
console.log("next_cursor:", page.next_cursor);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class GetResults {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.params['status'] = 'successful'
end
print(res.body)
```
```bash cURL theme={null}
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
To page through every result and download each body, loop over the pages until `next_cursor` is null, fetching each presigned `result_url` promptly.
```python Python theme={null}
## pip install requests
import os, requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
os.makedirs('out', exist_ok=True)
cursor = None
while True:
page = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results',
headers=headers, params={'status': 'successful', 'cursor': cursor}).json()
for row in page['results']:
# result_url is a presigned link, valid 2h, so fetch it promptly.
content = requests.get(row['result_url']).content
with open(f"out/{row.get('external_id') or row['task_id']}", 'wb') as f:
f.write(content)
cursor = page['next_cursor']
if not cursor:
break
```
```javascript Node.js theme={null}
async function* iterResults(jobId, status = "all") {
let cursor;
do {
const page = await api("GET", `/jobs/${jobId}/results`, { query: { status, cursor } });
yield* page.results;
cursor = page.next_cursor;
} while (cursor);
}
import { writeFile } from "node:fs/promises";
for await (const row of iterResults("JOB_ID", "successful")) {
// row.result_url is a presigned link, valid 2h, so fetch it promptly.
const bytes = Buffer.from(await (await fetch(row.result_url)).arrayBuffer());
await writeFile(`./out/${row.external_id ?? row.task_id}`, bytes);
}
```
```java Java theme={null}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import java.nio.file.Files;
import java.nio.file.Path;
public class DownloadResults {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String cursor = null;
do {
String url = "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful"
+ (cursor != null ? "&cursor=" + cursor : "");
JsonNode page = M.readTree(Request.get(url).addHeader("X-API-Key", key)
.execute().returnContent().asString());
for (JsonNode row : page.path("results")) {
// result_url is a presigned link, valid 2h, so fetch it promptly.
byte[] bytes = Request.get(row.path("result_url").asText())
.execute().returnContent().asBytes();
String name = row.path("external_id").asText(row.path("task_id").asText());
Files.write(Path.of("out", name), bytes);
}
cursor = page.path("next_cursor").asText(null);
} while (cursor != null && !cursor.isEmpty());
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"encoding/json"
"io"
"log"
"net/http"
"os"
"path/filepath"
)
func get(url, key string) []byte {
req, _ := http.NewRequest("GET", url, nil)
if key != "" {
req.Header.Set("X-API-Key", key)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return body
}
func main() {
key := "YOUR_ZENROWS_API_KEY"
os.MkdirAll("out", 0o755)
cursor := ""
for {
url := "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful"
if cursor != "" {
url += "&cursor=" + cursor
}
var page struct {
Results []struct {
TaskID string `json:"task_id"`
ExternalID string `json:"external_id"`
ResultURL string `json:"result_url"`
} `json:"results"`
NextCursor string `json:"next_cursor"`
}
json.Unmarshal(get(url, key), &page)
for _, row := range page.Results {
// result_url is a presigned link, valid 2h, so fetch it promptly.
name := row.ExternalID
if name == "" {
name = row.TaskID
}
os.WriteFile(filepath.Join("out", name), get(row.ResultURL, ""), 0o644)
}
cursor = page.NextCursor
if cursor == "" {
break
}
}
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
Dir.mkdir('out') unless Dir.exist?('out')
cursor = nil
loop do
res = Faraday.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/results') do |req|
req.headers['X-API-Key'] = key
req.params['status'] = 'successful'
req.params['cursor'] = cursor if cursor
end
page = JSON.parse(res.body)
page['results'].each do |row|
# result_url is a presigned link, valid 2h, so fetch it promptly.
name = row['external_id'] || row['task_id']
File.write("out/#{name}", Faraday.get(row['result_url']).body, mode: 'wb')
end
cursor = page['next_cursor']
break unless cursor
end
```
```bash cURL theme={null}
# Page with a shell loop, then download each presigned result_url.
mkdir -p out
cursor=""
while :; do
page=$(curl -s "https://async.api.zenrows.com/v1/jobs/JOB_ID/results?status=successful&cursor=$cursor" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY")
echo "$page" | jq -r '.results[] | [(.external_id // .task_id), .result_url] | @tsv' \
| while IFS=$'\t' read -r name url; do curl -s "$url" -o "out/$name"; done
cursor=$(echo "$page" | jq -r '.next_cursor // empty')
[ -z "$cursor" ] && break
done
```
You can also fetch one task's body directly with `GET /jobs/{job_id}/tasks/{task_id}/content`.
* `200`: returns the content
* `409`: means the task is still processing
* `422`: means the task failed and the body holds the stored error.
```python Python theme={null}
## pip install requests
import requests
# 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
response = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'})
print(response.status_code)
print(response.text)
```
```javascript Node.js theme={null}
// 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
const content = await api("GET", "/jobs/JOB_ID/tasks/TASK_ID/content");
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class TaskContent {
public static void main(final String... args) throws Exception {
// 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func main() {
// 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(resp.StatusCode, string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
# 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
conn = Faraday.new
res = conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
print(res.status)
print(res.body)
```
```bash cURL theme={null}
# 200 = content; 409 = still pending/processing; 422 = failed (body is the error)
curl -i "https://async.api.zenrows.com/v1/jobs/JOB_ID/tasks/TASK_ID/content" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
A failed task's results row already carries `row.error` (`{ code, detail, ... }`), so you rarely need the content endpoint to inspect a failure. During the beta, treat the error `code` as a hint: it does not yet reliably distinguish blocked, not-found, and timeout.
### Bulk export (ZIP)
Bundle an entire run's result bodies into a single ZIP. Start the export with `POST /jobs/{job_id}/runs/{run_id}/exports`, poll the export until `status` is `completed`, then download the presigned `download_url`.
Exports expire 12 hours after creation, the API presigns `download_url` fresh on each poll, and a run whose combined results exceed 1 GB fails with a clear error instead of producing a partial archive.
```python Python theme={null}
## pip install requests
import requests
started = requests.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'}).json()
print(started['export_id'], started['status'])
# Poll the export by id until status == 'completed', then download started['download_url'].
```
```javascript Node.js theme={null}
const started = await api("POST", "/jobs/JOB_ID/runs/RUN_ID/exports");
console.log(started.export_id, started.status);
// Poll the export by id until status === 'completed', then download download_url.
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class StartExport {
public static void main(final String... args) throws Exception {
String response = Request.post("https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response);
// Poll the export by id until status is "completed", then download download_url.
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
// Poll the export by id until status is "completed", then download download_url.
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
end
print(res.body)
# Poll the export by id until status is "completed", then download download_url.
```
```bash cURL theme={null}
# Start the export, then poll the returned export_id until status is "completed".
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports/EXPORT_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
Start the export, poll until it is `completed`, then download the ZIP.
```python Python theme={null}
## pip install requests
import time, requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
base = 'https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports'
exp = requests.post(base, headers=headers).json()
while exp['status'] not in ('completed', 'failed'):
time.sleep(3)
exp = requests.get(f"{base}/{exp['export_id']}", headers=headers).json()
if exp['status'] == 'failed':
raise RuntimeError(exp.get('error'))
with open('results.zip', 'wb') as f:
f.write(requests.get(exp['download_url']).content)
```
```javascript Node.js theme={null}
import { writeFile } from "node:fs/promises";
let exp = await api("POST", "/jobs/JOB_ID/runs/RUN_ID/exports");
while (exp.status !== "completed" && exp.status !== "failed") {
await new Promise((r) => setTimeout(r, 3000));
exp = await api("GET", `/jobs/JOB_ID/runs/RUN_ID/exports/${exp.export_id}`);
}
if (exp.status === "failed") throw new Error(exp.error);
const zip = Buffer.from(await (await fetch(exp.download_url)).arrayBuffer());
await writeFile("results.zip", zip);
```
```java Java theme={null}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import java.nio.file.Files;
import java.nio.file.Path;
public class ExportRun {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String base = "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports";
JsonNode exp = M.readTree(Request.post(base).addHeader("X-API-Key", key)
.execute().returnContent().asString());
String id = exp.path("export_id").asText();
while (!exp.path("status").asText().equals("completed") && !exp.path("status").asText().equals("failed")) {
Thread.sleep(3000);
exp = M.readTree(Request.get(base + "/" + id).addHeader("X-API-Key", key)
.execute().returnContent().asString());
}
if (exp.path("status").asText().equals("failed")) throw new RuntimeException(exp.path("error").asText());
byte[] zip = Request.get(exp.path("download_url").asText()).execute().returnContent().asBytes();
Files.write(Path.of("results.zip"), zip);
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"encoding/json"
"io"
"log"
"net/http"
"os"
"time"
)
func call(method, url, key string) map[string]any {
req, _ := http.NewRequest(method, url, nil)
if key != "" {
req.Header.Set("X-API-Key", key)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
var out map[string]any
json.Unmarshal(body, &out)
return out
}
func main() {
key := "YOUR_ZENROWS_API_KEY"
base := "https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports"
exp := call("POST", base, key)
id, _ := exp["export_id"].(string)
for exp["status"] != "completed" && exp["status"] != "failed" {
time.Sleep(3 * time.Second)
exp = call("GET", base+"/"+id, key)
}
if exp["status"] == "failed" {
log.Fatalln(exp["error"])
}
req, _ := http.NewRequest("GET", exp["download_url"].(string), nil)
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
out, _ := os.Create("results.zip")
defer out.Close()
io.Copy(out, resp.Body)
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
key = 'YOUR_ZENROWS_API_KEY'
base = 'https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports'
exp = JSON.parse(Faraday.post(base) { |r| r.headers['X-API-Key'] = key }.body)
until %w[completed failed].include?(exp['status'])
sleep 3
exp = JSON.parse(Faraday.get("#{base}/#{exp['export_id']}") { |r| r.headers['X-API-Key'] = key }.body)
end
raise(exp['error'] || 'export failed') if exp['status'] == 'failed'
File.write('results.zip', Faraday.get(exp['download_url']).body, mode: 'wb')
```
```bash cURL theme={null}
base="https://async.api.zenrows.com/v1/jobs/JOB_ID/runs/RUN_ID/exports"
started=$(curl -s -X POST "$base" -H "X-API-Key: YOUR_ZENROWS_API_KEY")
id=$(echo "$started" | jq -r '.export_id')
while :; do
exp=$(curl -s "$base/$id" -H "X-API-Key: YOUR_ZENROWS_API_KEY")
status=$(echo "$exp" | jq -r '.status')
case "$status" in completed|failed) break ;; esac
sleep 3
done
[ "$status" = "failed" ] && { echo "export failed"; exit 1; }
curl -s "$(echo "$exp" | jq -r '.download_url')" -o results.zip
```
## 4. Webhooks and notifications
Get notified when a run finishes, and secure those callbacks.
### Set up webhooks
Attach a webhook so the API POSTs a `run.completed` event to your HTTPS URL when a run finishes. Set `signature: true` for HMAC-signed deliveries (see [Manage HMAC keys](#manage-hmac-keys)).
You can attach it on the submit body, replace it later with `PUT /jobs/{id}/webhook`, remove it with `DELETE /jobs/{id}/webhook`, and probe a receiver with `POST /webhook/test`.
```python Python theme={null}
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'}
webhook = {'url': 'https://your-app.com/webhooks/zenrows', 'signature': True}
# Set or replace the webhook on an existing job:
requests.put('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook', headers=headers, data=json.dumps(webhook))
# Probe a receiver before wiring it up:
probe = requests.post('https://async.api.zenrows.com/v1/webhook/test', headers=headers, data=json.dumps(webhook)).json()
print(probe['delivered'], probe['status_code'])
# Stop deliveries:
requests.delete('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook', headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'})
```
```javascript Node.js theme={null}
const webhook = { url: "https://your-app.com/webhooks/zenrows", signature: true };
// At submit time, include it on the body: { ..., webhook }.
// Or manage it on an existing job:
await api("PUT", "/jobs/JOB_ID/webhook", { body: webhook });
await api("DELETE", "/jobs/JOB_ID/webhook"); // stop deliveries
// Verify a receiver before wiring it to a job:
const probe = await api("POST", "/webhook/test", { body: webhook });
console.log(probe.delivered, probe.status_code);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class Webhook {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String webhook = "{\"url\":\"https://your-app.com/webhooks/zenrows\",\"signature\":true}";
// Set or replace the webhook on an existing job:
Request.put("https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook")
.addHeader("X-API-Key", key).bodyString(webhook, ContentType.APPLICATION_JSON)
.execute().discardContent();
// Probe a receiver:
String probe = Request.post("https://async.api.zenrows.com/v1/webhook/test")
.addHeader("X-API-Key", key).bodyString(webhook, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(probe);
// Stop deliveries:
Request.delete("https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook")
.addHeader("X-API-Key", key).execute().discardContent();
}
}
```
```php PHP theme={null}
'https://your-app.com/webhooks/zenrows', 'signature' => true]);
// Set or replace the webhook on an existing job:
$ch = curl_init('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook');
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_POSTFIELDS, $webhook);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_exec($ch);
curl_close($ch);
// Probe a receiver:
$ch = curl_init('https://async.api.zenrows.com/v1/webhook/test');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $webhook);
curl_setopt($ch, CURLOPT_HTTPHEADER, ["X-API-Key: $key", 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
```
```go Go theme={null}
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
webhook := `{ "url": "https://your-app.com/webhooks/zenrows", "signature": true }`
// Set or replace the webhook on an existing job:
putReq, _ := http.NewRequest("PUT", "https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook", bytes.NewBufferString(webhook))
putReq.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
putReq.Header.Set("Content-Type", "application/json")
if _, err := http.DefaultClient.Do(putReq); err != nil {
log.Fatalln(err)
}
// Probe a receiver:
testReq, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/webhook/test", bytes.NewBufferString(webhook))
testReq.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
testReq.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(testReq)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
conn.headers['Content-Type'] = 'application/json'
webhook = { url: 'https://your-app.com/webhooks/zenrows', signature: true }.to_json
# Set or replace the webhook on an existing job:
conn.put('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook', webhook)
# Probe a receiver:
puts conn.post('https://async.api.zenrows.com/v1/webhook/test', webhook).body
# Stop deliveries:
conn.delete('https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook')
```
```bash cURL theme={null}
# Set or replace the webhook on an existing job:
curl -X PUT "https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "url": "https://your-app.com/webhooks/zenrows", "signature": true }'
# Probe a receiver:
curl -X POST "https://async.api.zenrows.com/v1/webhook/test" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "url": "https://your-app.com/webhooks/zenrows", "signature": false }'
# Stop deliveries:
curl -X DELETE "https://async.api.zenrows.com/v1/jobs/JOB_ID/webhook" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
The API delivers each event at least once, so deduplicate on the `X-ZenRows-Event-Id` header. Webhooks are a convenience; the run state and result URLs remain the reliable source of truth.
### Manage HMAC keys
Signed webhook deliveries need an active HMAC key for your organization. Rotation uses two slots, an active key plus a candidate, so you can roll keys without downtime.
* `POST /hmac/keys/rotate` creates the first key or stages a candidate (the secret is returned once)
* `POST /hmac/keys/rotate/finalize` promotes the candidate and retires the old key
* `DELETE /hmac/keys/rotate` discards a staged candidate
* `GET /hmac/keys` lists key metadata without secrets
```python Python theme={null}
## pip install requests
import requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
created = requests.post('https://async.api.zenrows.com/v1/hmac/keys/rotate', headers=headers).json()
print(created) # store created['secret'] now; it is returned only once
requests.post('https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize', headers=headers)
# requests.delete('https://async.api.zenrows.com/v1/hmac/keys/rotate', headers=headers) # discard candidate
print(requests.get('https://async.api.zenrows.com/v1/hmac/keys', headers=headers).json())
```
```javascript Node.js theme={null}
const created = await api("POST", "/hmac/keys/rotate");
console.log(created.kid, created.secret); // the secret is returned ONCE, so store it now
await api("POST", "/hmac/keys/rotate/finalize"); // promote candidate -> active, retire the old
// await api("DELETE", "/hmac/keys/rotate"); // or discard the candidate
console.log(await api("GET", "/hmac/keys")); // metadata only; never returns secrets
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class HmacKeys {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
String created = Request.post("https://async.api.zenrows.com/v1/hmac/keys/rotate")
.addHeader("X-API-Key", key).execute().returnContent().asString();
System.out.println(created); // store the secret now; it is returned only once
Request.post("https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize")
.addHeader("X-API-Key", key).execute().discardContent();
// Request.delete("https://async.api.zenrows.com/v1/hmac/keys/rotate")
// .addHeader("X-API-Key", key).execute().discardContent(); // discard candidate
String list = Request.get("https://async.api.zenrows.com/v1/hmac/keys")
.addHeader("X-API-Key", key).execute().returnContent().asString();
System.out.println(list);
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func call(method, path string) string {
req, _ := http.NewRequest(method, "https://async.api.zenrows.com/v1"+path, nil)
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return string(body)
}
func main() {
log.Println(call("POST", "/hmac/keys/rotate")) // store the secret now
call("POST", "/hmac/keys/rotate/finalize") // promote the candidate
// call("DELETE", "/hmac/keys/rotate") // or discard the candidate
log.Println(call("GET", "/hmac/keys")) // metadata only
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
puts conn.post('https://async.api.zenrows.com/v1/hmac/keys/rotate').body # store the secret now
conn.post('https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize') # promote the candidate
# conn.delete('https://async.api.zenrows.com/v1/hmac/keys/rotate') # or discard the candidate
puts conn.get('https://async.api.zenrows.com/v1/hmac/keys').body # metadata only
```
```bash cURL theme={null}
# Create the active key or stage a candidate (the secret is returned once):
curl -X POST "https://async.api.zenrows.com/v1/hmac/keys/rotate" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
# Promote the candidate to active and retire the old key:
curl -X POST "https://async.api.zenrows.com/v1/hmac/keys/rotate/finalize" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
# Or discard the staged candidate:
curl -X DELETE "https://async.api.zenrows.com/v1/hmac/keys/rotate" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
# List key metadata (never returns secrets):
curl "https://async.api.zenrows.com/v1/hmac/keys" -H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
### Verify signed webhooks
When `signature: true`, each delivery includes an `X-Signature` header in the form `t=,v1=,kid=`, where `t` is the timestamp, `v1` is the signature, and `kid` identifies the key used.
To verify a delivery:
1. Look up the base64 secret that matches the delivery's `kid`.
2. Compute `HMAC-SHA256(secret, ".")`, signing the timestamp, a literal dot, and the raw request body.
3. Compare the result to `v1`. If they match, the delivery is authentic.
Always sign the raw request body, never a re-serialized version, or the signature will not match. If `signature` is `false`, no `X-Signature` header is sent, so authenticate the callback another way, such as a secret path in the URL. Either way, deduplicate on the `X-ZenRows-Event-Id` header, since the same delivery can arrive more than once.
The examples below compute the expected signature so you can compare it to `v1` in your receiver.
```python Python theme={null}
import hmac, hashlib, base64
def verify(raw_body: bytes, t: str, v1: str, secret_b64: str) -> bool:
secret = base64.b64decode(secret_b64)
expected = hmac.new(secret, f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, v1)
```
```javascript Node.js theme={null}
import crypto from "node:crypto";
// SECRETS maps kid -> the base64 secret you captured at /hmac/keys/rotate.
function verifyWebhook(rawBody, signatureHeader, SECRETS) {
const parts = Object.fromEntries(
signatureHeader.split(",").map((kv) => kv.split("="))
); // { t, v1, kid }
const secret = SECRETS[parts.kid];
if (!secret) return false;
const expected = crypto
.createHmac("sha256", Buffer.from(secret, "base64"))
.update(`${parts.t}.${rawBody}`)
.digest("hex");
// constant-time compare
const a = Buffer.from(expected), b = Buffer.from(parts.v1);
return a.length === b.length && crypto.timingSafeEqual(a, b);
}
// Express-style receiver (capture the raw body!):
// app.post("/zr", express.raw({ type: "*/*" }), (req, res) => {
// const eventId = req.header("X-ZenRows-Event-Id"); // dedup on this
// const sig = req.header("X-Signature"); // absent => unsigned
// if (sig && !verifyWebhook(req.body.toString(), sig, SECRETS)) return res.sendStatus(401);
// if (alreadyHandled(eventId)) return res.sendStatus(200);
// handle(JSON.parse(req.body.toString()));
// res.sendStatus(200);
// });
```
```java Java theme={null}
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.Base64;
public class VerifyWebhook {
static boolean verify(String rawBody, String t, String v1, String secretB64) throws Exception {
byte[] secret = Base64.getDecoder().decode(secretB64);
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(secret, "HmacSHA256"));
byte[] digest = mac.doFinal((t + "." + rawBody).getBytes(StandardCharsets.UTF_8));
StringBuilder hex = new StringBuilder();
for (byte x : digest) hex.append(String.format("%02x", x));
return MessageDigest.isEqual(
hex.toString().getBytes(StandardCharsets.UTF_8), v1.getBytes(StandardCharsets.UTF_8));
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
)
func verify(rawBody, t, v1, secretB64 string) bool {
secret, _ := base64.StdEncoding.DecodeString(secretB64)
mac := hmac.New(sha256.New, secret)
mac.Write([]byte(t + "." + rawBody))
expected := hex.EncodeToString(mac.Sum(nil))
return hmac.Equal([]byte(expected), []byte(v1))
}
```
```ruby Ruby theme={null}
require 'openssl'
require 'base64'
def verify(raw_body, t, v1, secret_b64)
secret = Base64.decode64(secret_b64)
expected = OpenSSL::HMAC.hexdigest('SHA256', secret, "#{t}.#{raw_body}")
OpenSSL.fixed_length_secure_compare(expected, v1)
rescue StandardError
expected == v1
end
```
```bash cURL theme={null}
# Verification happens in your receiver, not in a request. With openssl:
# T and BODY come from the delivery; SECRET_B64 is your base64 key.
key_hex=$(printf '%s' "$SECRET_B64" | base64 -d | xxd -p -c 256)
printf '%s' "${T}.${BODY}" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$key_hex"
# Compare the printed hex digest against the v1 value from X-Signature.
```
## 5. Reruns and retrying failures
`POST /jobs/{job_id}/rerun` replays a job as a new run. Add `?status=failed` to re-run only the failures (append `,pending` to also include tasks that never started).
Already-successful tasks carry over, so you only re-scrape, and only pay for, what failed. The response reports `retried_tasks` and `inherited_tasks`.
```python Python theme={null}
## pip install requests
import requests
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY'}
# Full rerun:
requests.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun', headers=headers)
# Retry only the failures:
response = requests.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun',
headers=headers, params={'status': 'failed'})
data = response.json()
print('new run', data['latest_run']['run_id'], 'retried', data['retried_tasks'], 'inherited', data['inherited_tasks'])
```
```javascript Node.js theme={null}
// Full rerun: replay every task as a new run:
const full = await api("POST", "/jobs/JOB_ID/rerun");
// Retry only failures (add ",pending" to also re-run tasks that never started):
const retry = await api("POST", "/jobs/JOB_ID/rerun", { query: { status: "failed" } });
console.log("new run", retry.latest_run.run_id, "retried", retry.retried_tasks, "inherited", retry.inherited_tasks);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class Rerun {
public static void main(final String... args) throws Exception {
String key = "YOUR_ZENROWS_API_KEY";
// Full rerun:
Request.post("https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun")
.addHeader("X-API-Key", key).execute().discardContent();
// Retry only the failures:
String response = Request.post("https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun?status=failed")
.addHeader("X-API-Key", key).execute().returnContent().asString();
System.out.println(response); // includes retried_tasks and inherited_tasks
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func main() {
// Retry only the failures (drop the query string for a full rerun).
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun?status=failed", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body)) // includes retried_tasks and inherited_tasks
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
conn = Faraday.new
conn.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
# Full rerun:
conn.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun')
# Retry only the failures:
res = conn.post('https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun') do |req|
req.params['status'] = 'failed'
end
print(res.body) # includes retried_tasks and inherited_tasks
```
```bash cURL theme={null}
# Full rerun:
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
# Retry only the failures:
curl -X POST "https://async.api.zenrows.com/v1/jobs/JOB_ID/rerun?status=failed" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
## 6. Cost
Estimate spend before you submit, and read actual spend after a run.
### Estimate cost
Pricing is per successful request by tier, and the rate card is small and stable, so you can estimate cost on the client with no API call.
* Basic request (base)
* 5x multiplier (JavaScript rendering)
* 10x multiplier (premium proxy)
* 25x multiplier (both)
* a 1x to 25x range for `mode=auto`
Sum the tier across your tasks. The estimate assumes each URL succeeds once, so it is a single number unless a task uses `mode=auto`.
The estimate assumes each URL succeeds once, so it is a single number unless a task uses `mode=auto`. For more details on multipliers, see the [Pricing Page](/first-steps/pricing#cost-multipliers-for-universal-scraper-api).
```python Python theme={null}
def task_cost(params):
if str(params.get('mode')) == 'auto':
return (1, 25)
js = str(params.get('js_render')) == 'true'
proxy = str(params.get('premium_proxy')) == 'true'
c = 25 if js and proxy else 10 if proxy else 5 if js else 1
return (c, c)
def estimate(body):
job = body.get('zenrows_params', {})
lo = hi = 0
for t in body.get('tasks', []):
a, b = task_cost({**job, **t.get('zenrows_params', {})})
lo += a
hi += b
return {'tasks': len(body.get('tasks', [])), 'min': lo, 'max': hi, 'exact': lo == hi}
```
```javascript Node.js theme={null}
const TIERS = { base: 1, js: 5, proxy: 10, both: 25 }; // usage per successful request
function taskCost(params = {}) {
const v = (x) => String(x) === "true";
if (String(params.mode) === "auto") return { min: 1, max: 25 }; // dynamic
const js = v(params.js_render), proxy = v(params.premium_proxy);
const c = js && proxy ? TIERS.both : proxy ? TIERS.proxy : js ? TIERS.js : TIERS.base;
return { min: c, max: c };
}
function estimate(submitBody) {
const job = submitBody.zenrows_params ?? {};
let min = 0, max = 0;
for (const t of submitBody.tasks ?? []) {
const { min: a, max: b } = taskCost({ ...job, ...(t.zenrows_params ?? {}) });
min += a; max += b;
}
return { tasks: (submitBody.tasks ?? []).length, min, max, exact: min === max };
}
```
```java Java theme={null}
import java.util.List;
import java.util.Map;
public class Estimate {
static int[] taskCost(Map p) {
if ("auto".equals(p.get("mode"))) return new int[]{1, 25};
boolean js = "true".equals(p.get("js_render"));
boolean proxy = "true".equals(p.get("premium_proxy"));
int c = js && proxy ? 25 : proxy ? 10 : js ? 5 : 1;
return new int[]{c, c};
}
// Merge job-level and per-task params before calling taskCost for each task,
// then sum the min and max across all tasks.
static int[] estimate(List> mergedTaskParams) {
int min = 0, max = 0;
for (Map p : mergedTaskParams) {
int[] c = taskCost(p);
min += c[0];
max += c[1];
}
return new int[]{min, max};
}
}
```
```php PHP theme={null}
count($body['tasks'] ?? []), 'min' => $min, 'max' => $max, 'exact' => $min === $max];
}
?>
```
```go Go theme={null}
package main
func taskCost(p map[string]string) (int, int) {
if p["mode"] == "auto" {
return 1, 25
}
js := p["js_render"] == "true"
proxy := p["premium_proxy"] == "true"
switch {
case js && proxy:
return 25, 25
case proxy:
return 10, 10
case js:
return 5, 5
default:
return 1, 1
}
}
// Merge job-level and per-task params before calling taskCost for each task,
// then sum the min and max across all tasks.
func estimate(mergedTaskParams []map[string]string) (int, int) {
min, max := 0, 0
for _, p := range mergedTaskParams {
lo, hi := taskCost(p)
min += lo
max += hi
}
return min, max
}
```
```ruby Ruby theme={null}
def task_cost(params)
return [1, 25] if params['mode'].to_s == 'auto'
js = params['js_render'].to_s == 'true'
proxy = params['premium_proxy'].to_s == 'true'
c = js && proxy ? 25 : proxy ? 10 : js ? 5 : 1
[c, c]
end
def estimate(body)
job = body['zenrows_params'] || {}
min = 0; max = 0
(body['tasks'] || []).each do |t|
a, b = task_cost(job.merge(t['zenrows_params'] || {}))
min += a; max += b
end
{ tasks: (body['tasks'] || []).length, min: min, max: max, exact: min == max }
end
```
```bash cURL theme={null}
# Cost estimation is client-side arithmetic on the rate card, not an API call.
# Tiers (usage per successful request): base=1, js_render=5, premium_proxy=10, both=25, mode=auto=1..25.
# Sum the tier for each task. Example with jq over a submit body in body.json:
jq '[.tasks[] | (.zenrows_params // {}) as $p
| (if ($p.mode? == "auto") then 25
elif ($p.js_render? == "true" and $p.premium_proxy? == "true") then 25
elif ($p.premium_proxy? == "true") then 10
elif ($p.js_render? == "true") then 5
else 1 end)] | add' body.json
```
### Check actual cost
Once a run finishes, you can read the usage it actually consumed at two levels:
* **Run total:** the `spend` object on `latest_run.stats`, read from the `GET /jobs/{id}` response shown below.
* **Per task:** the `spend` object on each result row, where `spend.total.credits` is the cost across all attempts and `spend.last_attempt.credits` is the cost of the most recent attempt. Read these while paging results, as shown in [Download content](#download-content).
Only successful (status 200, 404 and 410) requests cost usage, so a run with failures costs less than its estimate. Treat this figure as an indicative signal. Your ZenRows account statement remains the authoritative record for billing.
```python Python theme={null}
## pip install requests
import requests
job = requests.get('https://async.api.zenrows.com/v1/jobs/JOB_ID',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY'}).json()
spend = job['latest_run']['stats'].get('spend')
if spend:
print('usage:', spend['credits'], 'cost:', spend['cost'])
```
```javascript Node.js theme={null}
const job = await api("GET", "/jobs/JOB_ID");
const spend = job.latest_run.stats.spend; // { credits, cost }, absent until there's spend
if (spend) console.log("usage:", spend.credits, "cost:", spend.cost);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
public class ActualCost {
public static void main(final String... args) throws Exception {
String response = Request.get("https://async.api.zenrows.com/v1/jobs/JOB_ID")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.execute().returnContent().asString();
System.out.println(response); // read latest_run.stats.spend from this JSON
}
}
```
```php PHP theme={null}
```
```go Go theme={null}
package main
import (
"io"
"log"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://async.api.zenrows.com/v1/jobs/JOB_ID", nil)
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(string(body)) // read latest_run.stats.spend from this JSON
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
job = JSON.parse(conn.get('https://async.api.zenrows.com/v1/jobs/JOB_ID') { |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
}.body)
spend = job.dig('latest_run', 'stats', 'spend')
puts "usage: #{spend['credits']} cost: #{spend['cost']}" if spend
```
```bash cURL theme={null}
# Read latest_run.stats.spend from the job response:
curl "https://async.api.zenrows.com/v1/jobs/JOB_ID" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY"
```
## 7. Reference
Behaviors and error details that apply across the API.
### Idempotency
Send an `Idempotency-Key` header on a submit to guard against accidentally creating duplicate jobs. Each key can be used once: the first request with a given key creates the job, and any later request that reuses the same key returns `409 idempotency_key_conflict`, whether or not the body is identical. The API does not store or replay the original response.
```python Python theme={null}
## pip install requests
import requests, json
headers = {'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json',
'Idempotency-Key': 'nightly-2026-06-08'}
payload = {'type': 'regular', 'status': 'closed',
'tasks': [{'url': 'https://www.scrapingcourse.com/ecommerce/'}]}
response = requests.post('https://async.api.zenrows.com/v1/jobs', headers=headers, data=json.dumps(payload))
print(response.status_code, response.json())
```
```javascript Node.js theme={null}
const job = await api("POST", "/jobs", {
headers: { "Idempotency-Key": "nightly-2026-06-08" },
body: { type: "regular", status: "closed", tasks: [{ url: "https://www.scrapingcourse.com/ecommerce/" }] },
});
console.log(job.job_id);
```
```java Java theme={null}
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ContentType;
public class Idempotent {
public static void main(final String... args) throws Exception {
String body = "{\"type\":\"regular\",\"status\":\"closed\",\"tasks\":[{\"url\":\"https://www.scrapingcourse.com/ecommerce/\"}]}";
String response = Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.addHeader("Idempotency-Key", "nightly-2026-06-08")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnContent().asString();
System.out.println(response);
}
}
```
```php PHP theme={null}
'regular', 'status' => 'closed',
'tasks' => [['url' => 'https://www.scrapingcourse.com/ecommerce/']],
]));
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'X-API-Key: YOUR_ZENROWS_API_KEY',
'Idempotency-Key: nightly-2026-06-08',
'Content-Type: application/json',
]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
echo curl_exec($ch) . PHP_EOL;
curl_close($ch);
?>
```
```go Go theme={null}
package main
import (
"bytes"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{"type":"regular","status":"closed","tasks":[{"url":"https://www.scrapingcourse.com/ecommerce/"}]}`)
req, err := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
if err != nil {
log.Fatalln(err)
}
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Idempotency-Key", "nightly-2026-06-08")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
log.Println(resp.StatusCode, string(body))
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
conn = Faraday.new
res = conn.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Idempotency-Key'] = 'nightly-2026-06-08'
req.headers['Content-Type'] = 'application/json'
req.body = { type: 'regular', status: 'closed',
tasks: [{ url: 'https://www.scrapingcourse.com/ecommerce/' }] }.to_json
end
print(res.status)
print(res.body)
```
```bash cURL theme={null}
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" \
-H "Idempotency-Key: nightly-2026-06-08" \
-H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "closed", "tasks": [{ "url": "https://www.scrapingcourse.com/ecommerce/" }] }'
```
An idempotency key is not a safe-retry token, because reusing it always returns `409`. If a submit fails midway, such as a dropped connection or a `503`, do not resend with the same key. First check whether the job was created by listing your recent jobs, then either use its `job_id` or submit again with a new key. A late `503` can leave a visible job with a `stopped` run, which is safe to `DELETE`.
### Error handling reference
Errors use RFC 7807 problem JSON with a stable `code` you can branch on. Task-level failures are not HTTP errors; they appear as `failed` rows in the results, each with its own `error` object. A successful `POST /jobs` returns `201 Created` (synchronous) or `202 Accepted` (asynchronous, for 10,000+ tasks or a CSV input); both carry the same response body.
| HTTP | `code` | Meaning |
| ---- | ------------------------------------------------------------ | ----------------------------------------------------------------------- |
| 400 | `invalid_argument` | Validation failed; `invalid_tasks[]` lists `{index, reason}`. |
| 401 | `unauthenticated` | Missing or invalid API key. |
| 402 | `payment_required` | No usage available. |
| 404 | `not_found` | Job, run, or task missing or not owned by you. |
| 409 | `conflict` / `run_not_terminal` / `idempotency_key_conflict` | State conflict; see `detail`. |
| 422 | (on `/content`) | Task failed; the body is the stored error. |
| 429 | `quota_exceeded` | Account limit reached, such as the maximum of 3 concurrent active jobs. |
| 503 | `internal` | Transient; safe to retry with a backoff. |
On a non-2xx response, read the `code` and, for a `400`, the `invalid_tasks` array from the problem body.
```python Python theme={null}
## pip install requests
import requests
r = requests.post('https://async.api.zenrows.com/v1/jobs',
headers={'X-API-Key': 'YOUR_ZENROWS_API_KEY', 'Content-Type': 'application/json'},
json={'type': 'regular', 'status': 'closed', 'tasks': [{'url': 'not-a-url'}]})
if not r.ok:
problem = r.json() # RFC 7807: { type, title, status, detail, code, invalid_tasks? }
print(r.status_code, problem.get('code'), problem.get('invalid_tasks'))
```
```javascript Node.js theme={null}
// The api() helper throws an object carrying the structured problem details.
try {
await api("POST", "/jobs", { body: { type: "regular", status: "closed", tasks: [{ url: "not-a-url" }] } });
} catch (e) {
console.error(e.status, e.code, e.problem?.invalid_tasks);
}
```
```java Java theme={null}
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.hc.client5.http.fluent.Request;
import org.apache.hc.core5.http.ClassicHttpResponse;
import org.apache.hc.core5.http.ContentType;
import org.apache.hc.core5.http.io.entity.EntityUtils;
public class ErrorHandling {
static final ObjectMapper M = new ObjectMapper();
public static void main(final String... args) throws Exception {
String body = "{\"type\":\"regular\",\"status\":\"closed\",\"tasks\":[{\"url\":\"not-a-url\"}]}";
ClassicHttpResponse resp = (ClassicHttpResponse) Request.post("https://async.api.zenrows.com/v1/jobs")
.addHeader("X-API-Key", "YOUR_ZENROWS_API_KEY")
.bodyString(body, ContentType.APPLICATION_JSON)
.execute().returnResponse();
if (resp.getCode() >= 400) {
JsonNode problem = M.readTree(EntityUtils.toString(resp.getEntity())); // RFC 7807
System.out.println(resp.getCode() + " " + problem.path("code").asText() + " " + problem.path("invalid_tasks"));
}
}
}
```
```php PHP theme={null}
'regular', 'status' => 'closed', 'tasks' => [['url' => 'not-a-url']]]));
curl_setopt($ch, CURLOPT_HTTPHEADER, ['X-API-Key: YOUR_ZENROWS_API_KEY', 'Content-Type: application/json']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if ($status >= 400) {
$problem = json_decode($res, true); // RFC 7807
echo $status, ' ', $problem['code'] ?? '', PHP_EOL;
print_r($problem['invalid_tasks'] ?? []);
}
?>
```
```go Go theme={null}
package main
import (
"bytes"
"encoding/json"
"io"
"log"
"net/http"
)
func main() {
payload := []byte(`{"type":"regular","status":"closed","tasks":[{"url":"not-a-url"}]}`)
req, _ := http.NewRequest("POST", "https://async.api.zenrows.com/v1/jobs", bytes.NewBuffer(payload))
req.Header.Set("X-API-Key", "YOUR_ZENROWS_API_KEY")
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode >= 400 {
var problem struct {
Code string `json:"code"`
InvalidTasks []any `json:"invalid_tasks"`
}
json.Unmarshal(body, &problem) // RFC 7807
log.Println(resp.StatusCode, problem.Code, problem.InvalidTasks)
}
}
```
```ruby Ruby theme={null}
# gem install faraday
require 'faraday'
require 'json'
res = Faraday.post('https://async.api.zenrows.com/v1/jobs') do |req|
req.headers['X-API-Key'] = 'YOUR_ZENROWS_API_KEY'
req.headers['Content-Type'] = 'application/json'
req.body = { type: 'regular', status: 'closed', tasks: [{ url: 'not-a-url' }] }.to_json
end
unless res.success?
problem = JSON.parse(res.body) # RFC 7807
puts "#{res.status} #{problem['code']}"
p problem['invalid_tasks']
end
```
```bash cURL theme={null}
# -i shows the status line; the body is the RFC 7807 problem with code and invalid_tasks.
curl -i -X POST "https://async.api.zenrows.com/v1/jobs" \
-H "X-API-Key: YOUR_ZENROWS_API_KEY" -H "Content-Type: application/json" \
-d '{ "type": "regular", "status": "closed", "tasks": [{ "url": "not-a-url" }] }'
```
## Troubleshooting
Common issues you may hit while integrating. See the full [Troubleshooting](/batch-scraper-api/troubleshooting) page for the complete list.
* **`401 unauthenticated`:** send your key in the `X-API-Key` header (never in the URL), and confirm your account has private-beta access.
* **`400 invalid_argument`:** the body failed validation. Read the `invalid_tasks` array to see which URL failed and why.
* **`429 quota_exceeded`:** you have 3 jobs running already. Wait for one to finish, since 3 concurrent jobs is the limit.
* **`503` when opening a job:** open (queue-mode) jobs are still being finalized during the beta. Use a closed job or a CSV upload for now.
* **Download link returns an error:** each `result_url` is valid for 2 hours. Re-list the results to get a fresh link rather than reusing a stored one.
* **Webhook never arrives:** use an HTTPS receiver, test it with `POST /webhook/test`, and deduplicate on the `X-ZenRows-Event-Id` header, since deliveries are at-least-once.
## FAQ (Frequently Asked Questions)
Quick answers for developers. See the full [FAQ](/batch-scraper-api/faq) for more.
All requests go to `https://async.api.zenrows.com/v1`, authenticated with the `X-API-Key` header. The Batch Scraper API uses the same API key as the Universal Scraper API.
Any language with an HTTP client. The examples in this guide cover Python, Node.js, Java, PHP, Go, Ruby, and cURL, and the same REST calls work anywhere.
Up to 100,000 URLs per job submit, 10,000 per add-tasks call on an open job, and 3 active jobs concurrently. CSV uploads support 50 MB / 100,000 rows.
No. Each key can be used once, and reusing it always returns `409 idempotency_key_conflict`. If a submit fails midway, check whether the job was created before resending with a new key.
In usage, per successful request, at the same rates as the Universal Scraper API. Failed requests are never charged.