> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> The Zenrows HTTP API serves Fetch and Extract (extract=auto) at https://api.zenrows.com/v1/, Crawl at https://api.zenrows.com/v1/crawls, and Batch at https://async.api.zenrows.com/v1. Fetch, Extract, Crawl and Batch are described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Every endpoint also has its own page under /api-reference with its schema. Use the spec for exact parameter names and types. Crawl's endpoints, parameters and errors are also at https://docs.zenrows.com/crawl/endpoints.md. To call Zenrows from an agent, connect the MCP server at https://mcp.zenrows.com/mcp. To search these docs from an agent, connect the docs MCP server at https://docs.zenrows.com/mcp. Agent skills for Zenrows are published at https://github.com/ZenRows/cli and install with `npx skills add ZenRows/cli`.

# Use different settings for each URL in one job

> Give each URL in one Zenrows Batch job its own settings, such as markdown for one page, selected fields for another and a country for a third.

One Batch job can mix very different pages. Set the common settings once at the job level, then give any task its own `zenrows_params`: markdown for a page you feed to an LLM, a few CSS-selected fields for a price page, a country for a localized page. Tag each task with `external_id` and `metadata`, and they come back on its result so your pipeline knows what to do with it.

**Uses:** [Batch `zenrows_params`](https://docs.zenrows.com/batch/developer-guide-restapi#configure-the-scrape-and-per-url-options) at job and task level, [`response_type`](https://docs.zenrows.com/fetch/features/markdown), [`css_extractor`](https://docs.zenrows.com/fetch/features/css-extractor), [`proxy_country`](https://docs.zenrows.com/fetch/features/proxy-country) and [Adaptive Stealth Mode](https://docs.zenrows.com/fetch/features/adaptive-stealth-mode.md) (`mode=auto`)

## Ask your AI agent

With the [Zenrows MCP server](https://docs.zenrows.com/mcp/overview.md) connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.

```text theme={"theme":"css-variables"}
Use Zenrows to scrape these three pages in one batch and wait for it to finish:
- https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/ as markdown
- https://www.scrapingcourse.com/ecommerce/product/aeon-capri/ returning only the product name and price
- https://httpbin.io/ip as a visitor from Germany
Then show me what came back for each one.
```

The same prompt works for thousands of URLs: describe the groups ("every product page as markdown, every category page as a list of links") and the agent sets the right options per URL.

## How job and task settings combine

* `zenrows_params` on the job applies to every task.
* `zenrows_params` on a task is merged on top of the job's: the task adds keys and replaces keys with the same name. Other tasks keep the job settings.
* `external_id` and `metadata` on a task are returned unchanged on that task's result.

So a job with `{"mode": "auto"}` and a task with `{"response_type": "markdown"}` runs that task with both.

## Submit a job with per-task settings

This job sets `mode=auto` for all tasks, then gives each task its own output:

```python Python (Requests) theme={"theme":"css-variables"}
import json
import time
import requests

BASE = "https://async.api.zenrows.com/v1"
headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"}

job = requests.post(
    f"{BASE}/jobs",
    headers=headers,
    json={
        "type": "regular",
        "status": "closed",
        # Job-level defaults: every task gets these unless it overrides them
        "zenrows_params": {"mode": "auto"},
        "tasks": [
            {   # 1. The page as markdown
                "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
                "external_id": "hoodie-markdown",
                "metadata": {"pipeline": "rag"},
                "zenrows_params": {"response_type": "markdown"},
            },
            {   # 2. Only the fields you need, as JSON
                "url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/",
                "external_id": "capri-fields",
                "metadata": {"pipeline": "prices"},
                "zenrows_params": {"css_extractor": json.dumps({"name": "h1", "price": "p.price span.amount"})},
            },
            {   # 3. Seen from Germany
                "url": "https://httpbin.io/ip",
                "external_id": "ip-de",
                "zenrows_params": {"proxy_country": "de"},
            },
        ],
    },
    timeout=60,
).json()
job_id = job["job_id"]
print("Submitted", job_id)

while True:
    run = requests.get(f"{BASE}/jobs/{job_id}", headers=headers, timeout=60).json()["latest_run"]
    if run["status"] in {"completed", "failed", "stopped"}:
        break
    time.sleep(5)
print(run["status"], f"{run['stats']['successful']}/{run['stats']['total']} successful")

page = requests.get(f"{BASE}/jobs/{job_id}/results", headers=headers, params={"status": "all"}, timeout=60).json()
for row in page["results"]:
    if row["status"] != "successful":
        print(row["external_id"], "FAILED", row["error"]["code"])
        continue
    body = requests.get(row["result_url"], timeout=60).text
    print(f"--- {row['external_id']} ({row['type']}) metadata={row.get('metadata')}")
    print(body[:300])
```

```javascript Node.js (SDK) theme={"theme":"css-variables"}
// npm install zenrows
import { ZenRowsBatchClient } from "zenrows";

const client = new ZenRowsBatchClient("YOUR_ZENROWS_API_KEY");

const job = await client.submitRegular(
  [
    {
      url: "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
      external_id: "hoodie-markdown",
      metadata: { pipeline: "rag" },
      zenrows_params: { response_type: "markdown" },
    },
    {
      url: "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/",
      external_id: "capri-fields",
      metadata: { pipeline: "prices" },
      zenrows_params: { css_extractor: JSON.stringify({ name: "h1", price: "p.price span.amount" }) },
    },
    {
      url: "https://httpbin.io/ip",
      external_id: "ip-de",
      zenrows_params: { proxy_country: "de" },
    },
  ],
  undefined,
  { zenrowsParams: { mode: "auto" } }, // job-level defaults
);

const run = await job.run.wait();
console.log(run.status, `${run.stats.successful}/${run.stats.total} successful`);

for await (const row of run.results({ status: "successful" })) {
  const body = (await run.downloadTaskToMemory(row)).toString();
  console.log(row.external_id, row.type, JSON.stringify(row.metadata), body.slice(0, 80).replace(/\n/g, " "));
}
```

```bash cURL theme={"theme":"css-variables"}
curl -X POST "https://async.api.zenrows.com/v1/jobs" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "regular",
    "status": "closed",
    "zenrows_params": { "mode": "auto" },
    "tasks": [
      { "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/", "external_id": "hoodie-markdown",
        "metadata": { "pipeline": "rag" }, "zenrows_params": { "response_type": "markdown" } },
      { "url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/", "external_id": "capri-fields",
        "metadata": { "pipeline": "prices" }, "zenrows_params": { "css_extractor": "{\"name\":\"h1\",\"price\":\"p.price span.amount\"}" } },
      { "url": "https://httpbin.io/ip", "external_id": "ip-de", "zenrows_params": { "proxy_country": "de" } }
    ]
  }'
```

```bash CLI theme={"theme":"css-variables"}
# tasks.jsonl: per-task settings sit next to the url on each line
# {"url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/", "external_id": "hoodie-markdown", "response_type": "markdown"}
# {"url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/", "external_id": "capri-fields", "css_extractor": {"name": "h1", "price": "p.price span.amount"}}
# {"url": "https://httpbin.io/ip", "external_id": "ip-de", "proxy_country": "de"}
zenrows batch create tasks.jsonl --follow
zenrows batch results YOUR_JOB_ID --download ./pages
```

`css_extractor` is a JSON object sent as a string, so encode it with `json.dumps` or `JSON.stringify`. The CLI encodes it for you and sends `mode=auto` on every task by default. In the CLI, any key on a line other than `url`, `external_id` and `metadata` becomes that task's setting.

## Example output

The Python run, trimmed, with the German IP address masked:

```text theme={"theme":"css-variables"}
Submitted 01M4ZCGP27063YY6S8TDJXKZP7
completed 3/3 successful
--- capri-fields (json) metadata={'pipeline': 'prices'}
{"name":"Aeon Capri","price":"$48.00"}
--- ip-de (html) metadata=None
{
  "origin": "203.0.113.24:45025"
}
--- hoodie-markdown (markdown) metadata={'pipeline': 'rag'}
[Skip to navigation](http://www.scrapingcourse.com#site-navigation) [Skip to content](http://www.scrapingcourse.com#content)
...
```

Each result's `type` follows its own settings: `markdown`, `json` for the CSS fields, `html` for the default. The CLI run:

```text theme={"theme":"css-variables"}
→ Submitting batch job (3 tasks, ~75 credits)…
✓ Submitted job 01M4Z1W52VA5PX1Z4X8EJDKH4G · status: completed
  3/3 completed · 3 successful · 0 failed
```

The CLI's credit figure is an upper bound. With `mode=auto` each task is billed for the configuration that worked.

## Mix Adaptive Stealth Mode with manual settings

To render one task with `js_render` and `wait_for` while the others use `mode=auto`, leave `mode` out of the job-level settings and set it on each task instead. A task's settings merge with the job's, so keep `mode=auto` and manual `js_render` or `premium_proxy` on separate tasks.

```python Python (Requests) theme={"theme":"css-variables"}
import time
import requests

BASE = "https://async.api.zenrows.com/v1"
headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"}

job_id = requests.post(f"{BASE}/jobs", headers=headers, json={
    "type": "regular",
    "status": "closed",
    # No job-level mode: each task picks its own
    "tasks": [
        {
            "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
            "external_id": "hoodie",
            "zenrows_params": {"mode": "auto", "response_type": "markdown"},
        },
        {
            "url": "https://www.scrapingcourse.com/javascript-rendering",
            "external_id": "js-grid",
            "zenrows_params": {"js_render": "true", "wait_for": ".product-item"},
        },
    ],
}, timeout=60).json()["job_id"]

while True:
    run = requests.get(f"{BASE}/jobs/{job_id}", headers=headers, timeout=60).json()["latest_run"]
    if run["status"] in {"completed", "failed", "stopped"}:
        break
    time.sleep(5)
print(run["status"], f"{run['stats']['successful']}/{run['stats']['total']} successful")
```

```text Output theme={"theme":"css-variables"}
completed 2/2 successful
```

## Customize it

| Change | How |
| - | - |
| Structured fields with no selectors | `"extract": "auto"` on the task, see [Get structured JSON from any page](https://docs.zenrows.com/academy/extract/auto-extraction.md) |
| Plain text for one task | `"response_type": "plaintext"`, see [Plain text](https://docs.zenrows.com/fetch/features/plaintext.md) |
| A PDF of one page | `"response_type": "pdf"` with `"js_render": "true"`, see [PDF](https://docs.zenrows.com/fetch/features/pdf.md). Batch doesn't take the `screenshot` parameters |
| Your own headers for one site | `"custom_headers": {"Referer": "https://www.google.com/"}`, a map of header name to value, see [Headers](https://docs.zenrows.com/fetch/features/headers.md) |
| Settings from your CSV | Upload the file and set job-level settings, see [Start a Batch job from a CSV file](https://docs.zenrows.com/academy/batch/csv-input.md) |
| Route results by tag | Read `metadata` or `external_id` on each result row and send it to the right pipeline |

## Troubleshooting

* **The job is rejected with `invalid_argument` and an `invalid_tasks` list**: Batch checks every task's settings before it starts. Each entry names the task `index` and the setting at fault: fix that setting and submit again.
* **A task fails with `REQS004` about `mode=auto`**: the task sets `js_render` or `premium_proxy` while the job sets `mode=auto`. Move `mode` from the job to the tasks that use it, as in [Mix Adaptive Stealth Mode with manual settings](#mix-adaptive-stealth-mode-with-manual-settings).
* **A `css_extractor` field comes back empty**: the selector doesn't match the page Zenrows received. Check it against the HTML of a plain run of that URL, and add `wait_for` if the element loads late.
* **`proxy_country` is rejected on a task**: use it with `mode=auto` or with `"premium_proxy": "true"` on the same task.

## Related recipes

> [card] [Scrape a list of URLs](https://docs.zenrows.com/academy/scrape-from-a-list-of-urls.md)
>
> Submit thousands of URLs as one Batch job.

> [card] [Extract data with CSS selectors](https://docs.zenrows.com/academy/how-to-extract-data.md)
>
> Write and test the selectors for your fields.

> [card] [Retry the tasks that failed](https://docs.zenrows.com/academy/batch/retry-failed-tasks.md)
>
> Rerun only the failures and keep the successes.

> [card] [Download all results of a job at once](https://docs.zenrows.com/academy/batch/export-results.md)
>
> Save every page to a folder or one ZIP file.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.