Use different settings for each URL in one job

One Batch job can mix very different pages. Set the common settings once at the job level, then give any task its own zenrows_params: markdown for a page you feed to an LLM, a few CSS-selected fields for a price page, a country for a localized page. Tag each task with external_id and metadata, and they come back on its result so your pipeline knows what to do with it. Uses: Batch zenrows_params at job and task level, response_type, css_extractor, proxy_country and Adaptive Stealth Mode (mode=auto)

Ask your AI agent

With the Zenrows MCP server connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.

Use Zenrows to scrape these three pages in one batch and wait for it to finish:
- https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/ as markdown
- https://www.scrapingcourse.com/ecommerce/product/aeon-capri/ returning only the product name and price
- https://httpbin.io/ip as a visitor from Germany
Then show me what came back for each one.

The same prompt works for thousands of URLs: describe the groups (“every product page as markdown, every category page as a list of links”) and the agent sets the right options per URL.

How job and task settings combine

zenrows_params on the job applies to every task.
zenrows_params on a task is merged on top of the job’s: the task adds keys and replaces keys with the same name. Other tasks keep the job settings.
external_id and metadata on a task are returned unchanged on that task’s result.

So a job with {"mode": "auto"} and a task with {"response_type": "markdown"} runs that task with both.

Submit a job with per-task settings

This job sets mode=auto for all tasks, then gives each task its own output:

import json
import time
import requests

BASE = "https://async.api.zenrows.com/v1"
headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"}

job = requests.post(
    f"{BASE}/jobs",
    headers=headers,
    json={
        "type": "regular",
        "status": "closed",
        # Job-level defaults: every task gets these unless it overrides them
        "zenrows_params": {"mode": "auto"},
        "tasks": [
            {   # 1. The page as markdown
                "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
                "external_id": "hoodie-markdown",
                "metadata": {"pipeline": "rag"},
                "zenrows_params": {"response_type": "markdown"},
            },
            {   # 2. Only the fields you need, as JSON
                "url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/",
                "external_id": "capri-fields",
                "metadata": {"pipeline": "prices"},
                "zenrows_params": {"css_extractor": json.dumps({"name": "h1", "price": "p.price span.amount"})},
            },
            {   # 3. Seen from Germany
                "url": "https://httpbin.io/ip",
                "external_id": "ip-de",
                "zenrows_params": {"proxy_country": "de"},
            },
        ],
    },
    timeout=60,
).json()
job_id = job["job_id"]
print("Submitted", job_id)

while True:
    run = requests.get(f"{BASE}/jobs/{job_id}", headers=headers, timeout=60).json()["latest_run"]
    if run["status"] in {"completed", "failed", "stopped"}:
        break
    time.sleep(5)
print(run["status"], f"{run['stats']['successful']}/{run['stats']['total']} successful")

page = requests.get(f"{BASE}/jobs/{job_id}/results", headers=headers, params={"status": "all"}, timeout=60).json()
for row in page["results"]:
    if row["status"] != "successful":
        print(row["external_id"], "FAILED", row["error"]["code"])
        continue
    body = requests.get(row["result_url"], timeout=60).text
    print(f"--- {row['external_id']} ({row['type']}) metadata={row.get('metadata')}")
    print(body[:300])

// npm install zenrows
import { ZenRowsBatchClient } from "zenrows";

const client = new ZenRowsBatchClient("YOUR_ZENROWS_API_KEY");

const job = await client.submitRegular(
  [
    {
      url: "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
      external_id: "hoodie-markdown",
      metadata: { pipeline: "rag" },
      zenrows_params: { response_type: "markdown" },
    },
    {
      url: "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/",
      external_id: "capri-fields",
      metadata: { pipeline: "prices" },
      zenrows_params: { css_extractor: JSON.stringify({ name: "h1", price: "p.price span.amount" }) },
    },
    {
      url: "https://httpbin.io/ip",
      external_id: "ip-de",
      zenrows_params: { proxy_country: "de" },
    },
  ],
  undefined,
  { zenrowsParams: { mode: "auto" } }, // job-level defaults
);

const run = await job.run.wait();
console.log(run.status, `${run.stats.successful}/${run.stats.total} successful`);

for await (const row of run.results({ status: "successful" })) {
  const body = (await run.downloadTaskToMemory(row)).toString();
  console.log(row.external_id, row.type, JSON.stringify(row.metadata), body.slice(0, 80).replace(/\n/g, " "));
}

curl -X POST "https://async.api.zenrows.com/v1/jobs" \
  -H "X-API-Key: YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "regular",
    "status": "closed",
    "zenrows_params": { "mode": "auto" },
    "tasks": [
      { "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/", "external_id": "hoodie-markdown",
        "metadata": { "pipeline": "rag" }, "zenrows_params": { "response_type": "markdown" } },
      { "url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/", "external_id": "capri-fields",
        "metadata": { "pipeline": "prices" }, "zenrows_params": { "css_extractor": "{\"name\":\"h1\",\"price\":\"p.price span.amount\"}" } },
      { "url": "https://httpbin.io/ip", "external_id": "ip-de", "zenrows_params": { "proxy_country": "de" } }
    ]
  }'

# tasks.jsonl: per-task settings sit next to the url on each line
# {"url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/", "external_id": "hoodie-markdown", "response_type": "markdown"}
# {"url": "https://www.scrapingcourse.com/ecommerce/product/aeon-capri/", "external_id": "capri-fields", "css_extractor": {"name": "h1", "price": "p.price span.amount"}}
# {"url": "https://httpbin.io/ip", "external_id": "ip-de", "proxy_country": "de"}
zenrows batch create tasks.jsonl --follow
zenrows batch results YOUR_JOB_ID --download ./pages

css_extractor is a JSON object sent as a string, so encode it with json.dumps or JSON.stringify. The CLI encodes it for you and sends mode=auto on every task by default. In the CLI, any key on a line other than url, external_id and metadata becomes that task’s setting.

Example output

The Python run, trimmed, with the German IP address masked:

Submitted 01M4ZCGP27063YY6S8TDJXKZP7
completed 3/3 successful
--- capri-fields (json) metadata={'pipeline': 'prices'}
{"name":"Aeon Capri","price":"$48.00"}
--- ip-de (html) metadata=None
{
  "origin": "203.0.113.24:45025"
}
--- hoodie-markdown (markdown) metadata={'pipeline': 'rag'}
[Skip to navigation](http://www.scrapingcourse.com#site-navigation) [Skip to content](http://www.scrapingcourse.com#content)
...

Each result’s type follows its own settings: markdown, json for the CSS fields, html for the default. The CLI run:

→ Submitting batch job (3 tasks, ~75 credits)…
✓ Submitted job 01M4Z1W52VA5PX1Z4X8EJDKH4G · status: completed
  3/3 completed · 3 successful · 0 failed

The CLI’s credit figure is an upper bound. With mode=auto each task is billed for the configuration that worked.

Mix Adaptive Stealth Mode with manual settings

To render one task with js_render and wait_for while the others use mode=auto, leave mode out of the job-level settings and set it on each task instead. A task’s settings merge with the job’s, so keep mode=auto and manual js_render or premium_proxy on separate tasks.

Python (Requests)

import time
import requests

BASE = "https://async.api.zenrows.com/v1"
headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"}

job_id = requests.post(f"{BASE}/jobs", headers=headers, json={
    "type": "regular",
    "status": "closed",
    # No job-level mode: each task picks its own
    "tasks": [
        {
            "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
            "external_id": "hoodie",
            "zenrows_params": {"mode": "auto", "response_type": "markdown"},
        },
        {
            "url": "https://www.scrapingcourse.com/javascript-rendering",
            "external_id": "js-grid",
            "zenrows_params": {"js_render": "true", "wait_for": ".product-item"},
        },
    ],
}, timeout=60).json()["job_id"]

while True:
    run = requests.get(f"{BASE}/jobs/{job_id}", headers=headers, timeout=60).json()["latest_run"]
    if run["status"] in {"completed", "failed", "stopped"}:
        break
    time.sleep(5)
print(run["status"], f"{run['stats']['successful']}/{run['stats']['total']} successful")

Output

completed 2/2 successful

Customize it

Change	How
Structured fields with no selectors	`"extract": "auto"` on the task, see Get structured JSON from any page
Plain text for one task	`"response_type": "plaintext"`, see Plain text
A screenshot of one page	`"screenshot": "true"` with `"js_render": "true"`, see Screenshot
Your own headers for one site	`"custom_headers": "true"`, see Headers
Settings from your CSV	Upload the file and set job-level settings, see Start a Batch job from a CSV file
Route results by tag	Read `metadata` or `external_id` on each result row and send it to the right pipeline

Troubleshooting

The job is rejected with invalid_argument and an invalid_tasks list: Batch checks every task’s settings before it starts. Each entry names the task index and the setting at fault: fix that setting and submit again.
A task fails with REQS004 about mode=auto: the task sets js_render or premium_proxy while the job sets mode=auto. Move mode from the job to the tasks that use it, as in Mix Adaptive Stealth Mode with manual settings.
A css_extractor field comes back empty: the selector doesn’t match the page Zenrows received. Check it against the HTML of a plain run of that URL, and add wait_for if the element loads late.
proxy_country is rejected on a task: use it with mode=auto or with "premium_proxy": "true" on the same task.

Scrape a list of URLs

Submit thousands of URLs as one Batch job.

Extract data with CSS selectors

Write and test the selectors for your fields.

Retry the tasks that failed

Rerun only the failures and keep the successes.

Download all results of a job at once

Save every page to a folder or one ZIP file.

Start Here

Foundations Course

Fetch Recipes

Extract Recipes

Crawl Recipes

Batch Recipes

Browser Recipes

Pipelines

Use Cases

By Surface

Production

Use different settings for each URL in one job

Ask your AI agent

How job and task settings combine

Submit a job with per-task settings

Example output

Mix Adaptive Stealth Mode with manual settings

Customize it

Troubleshooting

Scrape a list of URLs

Extract data with CSS selectors

Retry the tasks that failed

Download all results of a job at once

​Ask your AI agent

​How job and task settings combine

​Submit a job with per-task settings

​Example output

​Mix Adaptive Stealth Mode with manual settings

​Customize it

​Troubleshooting

​Related recipes

Scrape a list of URLs

Extract data with CSS selectors

Retry the tasks that failed

Download all results of a job at once

Ask your AI agent

How job and task settings combine

Submit a job with per-task settings

Example output

Mix Adaptive Stealth Mode with manual settings

Customize it

Troubleshooting

Related recipes