> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> The Zenrows API (Fetch, Extract via extract=auto, and Batch) is described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Use the spec for exact parameter names and types.

# Use different settings for each URL in one job

> Give each URL in one Zenrows Batch job its own settings, such as markdown for one page, selected fields for another and a country for a third.

One Batch job can mix very different pages. Set the common settings once at the job level, then give any task its own `zenrows_params`: markdown for a page you feed to an LLM, a few CSS-selected fields for a price page, a country for a localized page. Tag each task with `external_id` and `metadata`, and they come back on its result so your pipeline knows what to do with it.

**Uses:** [Batch `zenrows_params`](/batch/developer-guide-restapi#configure-the-scrape-and-per-url-options) at job and task level, [`response_type`](/fetch/features/markdown), [`css_extractor`](/fetch/features/css-extractor), [`proxy_country`](/fetch/features/proxy-country) and [Adaptive Stealth Mode](/fetch/features/adaptive-stealth-mode) (`mode=auto`)

## Ask your AI agent

With the [Zenrows MCP server](/mcp/overview) connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.

```text theme={"dark"}
Use Zenrows to scrape these three pages in one batch and wait for it to finish:
- https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/ as markdown
- https://www.scrapingcourse.com/ecommerce/product/aeon-capri/ returning only the product name and price
- https://httpbin.io/ip as a visitor from Germany
Then show me what came back for each one.
```

The same prompt works for thousands of URLs: describe the groups ("every product page as markdown, every category page as a list of links") and the agent sets the right options per URL.

## How job and task settings combine

* `zenrows_params` on the job applies to every task.
* `zenrows_params` on a task is merged on top of the job's: the task adds keys and replaces keys with the same name. Other tasks keep the job settings.
* `external_id` and `metadata` on a task are returned unchanged on that task's result.

So a job with `{"mode": "auto"}` and a task with `{"response_type": "markdown"}` runs that task with both.

## Submit a job with per-task settings

This job sets `mode=auto` for all tasks, then gives each task its own output:

<CodeGroup>
  ```python Python (Requests) theme={"dark"}
  import json
  import time
  import requests

  BASE = "https://async.api.zenrows.com/v1"
  headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"}

  job = requests.post(
      f"{BASE}/jobs",
      headers=headers,
      json={
          "type": "regular",
          "status": "closed",
          # Job-level defaults: every task gets these unless it overrides them
          "zenrows_params": {"mode": "auto"},
          "tasks": [
              {   # 1. The page as markdown
                  "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
                  "external_id": "hoodie-markdown",
                  "metadata": {"pipeline": "rag"},
                  "zenrows_params": {"response_type": "markdown"},
              },
              {   # 2. Only the fields you need, as JSON
                  "url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/",
                  "external_id": "capri-fields",
                  "metadata": {"pipeline": "prices"},
                  "zenrows_params": {"css_extractor": json.dumps({"name": "h1", "price": "p.price span.amount"})},
              },
              {   # 3. Seen from Germany
                  "url": "https://httpbin.io/ip",
                  "external_id": "ip-de",
                  "zenrows_params": {"proxy_country": "de"},
              },
          ],
      },
      timeout=60,
  ).json()
  job_id = job["job_id"]
  print("Submitted", job_id)

  while True:
      run = requests.get(f"{BASE}/jobs/{job_id}", headers=headers, timeout=60).json()["latest_run"]
      if run["status"] in {"completed", "failed", "stopped"}:
          break
      time.sleep(5)
  print(run["status"], f"{run['stats']['successful']}/{run['stats']['total']} successful")

  page = requests.get(f"{BASE}/jobs/{job_id}/results", headers=headers, params={"status": "all"}, timeout=60).json()
  for row in page["results"]:
      if row["status"] != "successful":
          print(row["external_id"], "FAILED", row["error"]["code"])
          continue
      body = requests.get(row["result_url"], timeout=60).text
      print(f"--- {row['external_id']} ({row['type']}) metadata={row.get('metadata')}")
      print(body[:300])
  ```

  ```javascript Node.js (SDK) theme={"dark"}
  // npm install zenrows
  import { ZenRowsBatchClient } from "zenrows";

  const client = new ZenRowsBatchClient("YOUR_ZENROWS_API_KEY");

  const job = await client.submitRegular(
    [
      {
        url: "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
        external_id: "hoodie-markdown",
        metadata: { pipeline: "rag" },
        zenrows_params: { response_type: "markdown" },
      },
      {
        url: "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/",
        external_id: "capri-fields",
        metadata: { pipeline: "prices" },
        zenrows_params: { css_extractor: JSON.stringify({ name: "h1", price: "p.price span.amount" }) },
      },
      {
        url: "https://httpbin.io/ip",
        external_id: "ip-de",
        zenrows_params: { proxy_country: "de" },
      },
    ],
    undefined,
    { zenrowsParams: { mode: "auto" } }, // job-level defaults
  );

  const run = await job.run.wait();
  console.log(run.status, `${run.stats.successful}/${run.stats.total} successful`);

  for await (const row of run.results({ status: "successful" })) {
    const body = (await run.downloadTaskToMemory(row)).toString();
    console.log(row.external_id, row.type, JSON.stringify(row.metadata), body.slice(0, 80).replace(/\n/g, " "));
  }
  ```

  ```bash cURL theme={"dark"}
  curl -X POST "https://async.api.zenrows.com/v1/jobs" \
    -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "type": "regular",
      "status": "closed",
      "zenrows_params": { "mode": "auto" },
      "tasks": [
        { "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/", "external_id": "hoodie-markdown",
          "metadata": { "pipeline": "rag" }, "zenrows_params": { "response_type": "markdown" } },
        { "url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/", "external_id": "capri-fields",
          "metadata": { "pipeline": "prices" }, "zenrows_params": { "css_extractor": "{\"name\":\"h1\",\"price\":\"p.price span.amount\"}" } },
        { "url": "https://httpbin.io/ip", "external_id": "ip-de", "zenrows_params": { "proxy_country": "de" } }
      ]
    }'
  ```

  ```bash CLI theme={"dark"}
  # tasks.jsonl: per-task settings sit next to the url on each line
  # {"url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/", "external_id": "hoodie-markdown", "response_type": "markdown"}
  # {"url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/", "external_id": "capri-fields", "css_extractor": {"name": "h1", "price": "p.price span.amount"}}
  # {"url": "https://httpbin.io/ip", "external_id": "ip-de", "proxy_country": "de"}
  zenrows batch create tasks.jsonl --follow
  zenrows batch results YOUR_JOB_ID --download ./pages
  ```
</CodeGroup>

`css_extractor` is a JSON object sent as a string, so encode it with `json.dumps` or `JSON.stringify`. The CLI encodes it for you and sends `mode=auto` on every task by default. In the CLI, any key on a line other than `url`, `external_id` and `metadata` becomes that task's setting.

## Example output

The Python run, trimmed, with the German IP address masked:

```text theme={"dark"}
Submitted 01M4ZCGP27063YY6S8TDJXKZP7
completed 3/3 successful
--- capri-fields (json) metadata={'pipeline': 'prices'}
{"name":"Aeon Capri","price":"$48.00"}
--- ip-de (html) metadata=None
{
  "origin": "203.0.113.24:45025"
}
--- hoodie-markdown (markdown) metadata={'pipeline': 'rag'}
[Skip to navigation](http://www.scrapingcourse.com#site-navigation) [Skip to content](http://www.scrapingcourse.com#content)
...
```

Each result's `type` follows its own settings: `markdown`, `json` for the CSS fields, `html` for the default. The CLI run:

```text theme={"dark"}
→ Submitting batch job (3 tasks, ~75 credits)…
✓ Submitted job 01M4Z1W52VA5PX1Z4X8EJDKH4G · status: completed
  3/3 completed · 3 successful · 0 failed
```

The CLI's credit figure is an upper bound. With `mode=auto` each task is billed for the configuration that worked.

## Mix Adaptive Stealth Mode with manual settings

To render one task with `js_render` and `wait_for` while the others use `mode=auto`, leave `mode` out of the job-level settings and set it on each task instead. A task's settings merge with the job's, so keep `mode=auto` and manual `js_render` or `premium_proxy` on separate tasks.

```python Python (Requests) theme={"dark"}
import time
import requests

BASE = "https://async.api.zenrows.com/v1"
headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"}

job_id = requests.post(f"{BASE}/jobs", headers=headers, json={
    "type": "regular",
    "status": "closed",
    # No job-level mode: each task picks its own
    "tasks": [
        {
            "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
            "external_id": "hoodie",
            "zenrows_params": {"mode": "auto", "response_type": "markdown"},
        },
        {
            "url": "https://www.scrapingcourse.com/javascript-rendering",
            "external_id": "js-grid",
            "zenrows_params": {"js_render": "true", "wait_for": ".product-item"},
        },
    ],
}, timeout=60).json()["job_id"]

while True:
    run = requests.get(f"{BASE}/jobs/{job_id}", headers=headers, timeout=60).json()["latest_run"]
    if run["status"] in {"completed", "failed", "stopped"}:
        break
    time.sleep(5)
print(run["status"], f"{run['stats']['successful']}/{run['stats']['total']} successful")
```

```text Output theme={"dark"}
completed 2/2 successful
```

## Customize it

| Change | How |
| - | - |
| Structured fields with no selectors | `"extract": "auto"` on the task, see [Get structured JSON from any page](/academy/extract/auto-extraction) |
| Plain text for one task | `"response_type": "plaintext"`, see [Plain text](/fetch/features/plaintext) |
| A screenshot of one page | `"screenshot": "true"` with `"js_render": "true"`, see [Screenshot](/fetch/features/screenshot) |
| Your own headers for one site | `"custom_headers": "true"`, see [Headers](/fetch/features/headers) |
| Settings from your CSV | Upload the file and set job-level settings, see [Start a Batch job from a CSV file](/academy/batch/csv-input) |
| Route results by tag | Read `metadata` or `external_id` on each result row and send it to the right pipeline |

## Troubleshooting

* **The job is rejected with `invalid_argument` and an `invalid_tasks` list**: Batch checks every task's settings before it starts. Each entry names the task `index` and the setting at fault: fix that setting and submit again.
* **A task fails with `REQS004` about `mode=auto`**: the task sets `js_render` or `premium_proxy` while the job sets `mode=auto`. Move `mode` from the job to the tasks that use it, as in [Mix Adaptive Stealth Mode with manual settings](#mix-adaptive-stealth-mode-with-manual-settings).
* **A `css_extractor` field comes back empty**: the selector doesn't match the page Zenrows received. Check it against the HTML of a plain run of that URL, and add `wait_for` if the element loads late.
* **`proxy_country` is rejected on a task**: use it with `mode=auto` or with `"premium_proxy": "true"` on the same task.

## Related recipes

<CardGroup cols={2}>
  <Card title="Scrape a list of URLs" icon="list" href="/academy/scrape-from-a-list-of-urls">
    Submit thousands of URLs as one Batch job.
  </Card>

  <Card title="Extract data with CSS selectors" icon="code" href="/academy/how-to-extract-data">
    Write and test the selectors for your fields.
  </Card>

  <Card title="Retry the tasks that failed" icon="refresh" href="/academy/batch/retry-failed-tasks">
    Rerun only the failures and keep the successes.
  </Card>

  <Card title="Download all results of a job at once" icon="download" href="/academy/batch/export-results">
    Save every page to a folder or one ZIP file.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.