Automate scraping from your terminal with the CLI

The zenrows CLI wraps every Zenrows primitive in one command, writes clean output to stdout and progress to stderr, and keeps a trace of every run. That makes it a natural fit for shell scripts, jq pipelines and cron jobs. Uses: zenrows fetch, zenrows extract, zenrows batch, zenrows crawl, zenrows browser, traces

Ask your AI agent

With the Zenrows MCP server connected, or the CLI available to a coding agent such as Claude Code, paste this prompt:

Use Zenrows to find the product pages on https://www.scrapingcourse.com/ecommerce/ (up to 10 page loads),
scrape them all in one batch as markdown, and save each page to a folder called pages/.

Install and log in

Install the CLI once, then set up a workspace with your API key. zenrows init creates the local .zenrows/ folder that holds your config, policy and run traces.

CLI

npm install -g @zenrows/cli
zenrows init --yes --api-key YOUR_ZENROWS_API_KEY

In CI or containers, export the key instead and tell the CLI to read it:

CLI

export ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY
zenrows login --env

Fetch and extract into jq

zenrows fetch and zenrows extract use Adaptive Stealth Mode (mode=auto) by default. The page body goes to stdout and the status line goes to stderr, so pipes stay clean. zenrows extract returns structured JSON, so you can pipe it straight into jq. With --css, you name the fields and their selectors:

CLI

zenrows extract https://www.scrapingcourse.com/ecommerce/ \
  --css '{"name":".product-name","price":".product .price"}' \
  | jq '[.name, .price] | transpose | map({name: .[0], price: .[1]}) | .[:3]'

[
  { "name": "Abominable Hoodie", "price": "$69.00" },
  { "name": "Adrienne Trek Jacket", "price": "$57.00" },
  { "name": "Aeon Capri", "price": "$48.00" }
]

Without --css, zenrows extract sends extract=auto for site-tailored fields on domains Zenrows has prepared, such as major retailers, and uses Autoparse on other domains. Save any page as markdown with --output markdown and --out:

CLI

zenrows fetch https://www.scrapingcourse.com/ecommerce/ --output markdown --out catalog.md

→ Protected Fetch https://www.scrapingcourse.com/ecommerce/ (auto)…
✓ Wrote 6763 bytes → catalog.md
✓ HTTP 200 · 6763 bytes · cost $0.0010 · 1 credit · run r-1a9dfd1e

Script with —json

Every command accepts --json and prints a structured result instead of the body or the status line. Use it to branch on status or log cost in a script:

CLI

zenrows fetch https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/ --json \
  | jq '{ok, httpStatus, costCredits, runId}'

{
  "ok": true,
  "httpStatus": 200,
  "costCredits": 1,
  "runId": "r-bcdfc63b"
}

Run a URL list as one Batch job

For tens to thousands of URLs, submit them as one Batch job instead of a loop of fetches. The CLI reads a JSONL file with one {"url": ...} object per line.

Build the job file from a plain URL list

CLI

jq -Rc '{url: .}' urls.txt > jobs.jsonl

Estimate the cost locally

estimate validates the file and prints an upper bound, with no cloud call and no credits spent. Tasks in Adaptive Stealth Mode count at their highest possible cost.

CLI

zenrows batch estimate jobs.jsonl

Job spec: jobs.jsonl
valid jobs: 3/3
estimated credits, upper bound (1x basic / 5x js / 10x proxy / 25x both or auto): 75
✓ Spec is valid.

Submit and wait for the run

--follow polls until the run finishes. --output markdown sets the format of every result.

CLI

zenrows batch create jobs.jsonl --output markdown --follow

→ Submitting batch job (3 tasks, ~75 credits)…
✓ Submitted job 01M4Y9ACBYZZFWWBY7JA8KE6QQ · status: completed
  3/3 completed · 3 successful · 0 failed

Download every result

CLI

zenrows batch results 01M4Y9ACBYZZFWWBY7JA8KE6QQ --download pages

pages/ gets one file per task plus _manifest.jsonl, which maps each file to its URL and status:

{"task_id":"01M4Y9ACBYS5SHV7V54EGYVW2D","url":"https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/","status":"successful","type":"markdown","file":"01M4Y9ACBYS5SHV7V54EGYVW2D.md"}

The run above billed 3 credits in total: the estimate is a ceiling, and mode=auto bills only the configuration that succeeds. Check the real spend with zenrows batch status <job-id> --json, which returns stats.spend.credits.

Crawl a site, then batch what you found

zenrows crawl create discovers URLs from one start page. Feed the results straight into a Batch job:

CLI

zenrows crawl create https://www.scrapingcourse.com/ecommerce/ \
  --max-pages 10 --include-patterns /product/ --include-patterns /page/ --follow

zenrows crawl results <crawl-id> --out crawl.jsonl
jq -c 'select(.url | contains("/product/")) | {url}' crawl.jsonl > jobs.jsonl

→ Starting crawl of https://www.scrapingcourse.com/ecommerce/ (no depth limit)…
✓ Crawl c_01M4Y9B3GA71076Z7S3MZ2XGP7 finished · status: completed (max_items)
  100 item(s) found · 10 page(s) fetched · 0 failed

Including /page/ lets the crawl walk the paginated listing, and the jq filter keeps only product URLs (88 of the 100 in this run). See Crawl from a seed URL for more patterns.

Drive a browser from the shell

When a page needs clicks or typing, open a Browser session. Each verb takes the --session id that open returns. Always close the session when you finish.

CLI

SESSION=$(zenrows browser open https://www.scrapingcourse.com/button-click --json | jq -r .session_id)

zenrows browser click --session "$SESSION" --selector "#load-more-btn"
zenrows browser wait --session "$SESSION" --ms 2000
zenrows browser query-selector-all --session "$SESSION" --selector ".product-name" --json \
  | jq -r '.result.elements[].text'

zenrows browser close --session "$SESSION"

Chaz Kangeroo Hoodie
Teton Pullover Hoodie
Bruno Compete Hoodie
Frankie Sweatshirt

For longer flows, put the steps in a JSON file and run them with zenrows browser run script.json, which closes the session for you. See the Browser CLI guide.

Schedule it with cron

The CLI keeps your key and policy in .zenrows/, so a cron job only needs to run from that folder. This script refreshes the product pages every morning:

scrape.sh

#!/usr/bin/env bash
set -euo pipefail
cd /home/me/catalog

JOB=$(zenrows batch create jobs.jsonl --output markdown --follow --json | jq -r .jobId)
zenrows batch results "$JOB" --download "pages/$(date +%F)"

crontab

0 6 * * * /home/me/catalog/scrape.sh >> /home/me/catalog/scrape.log 2>&1

For schedules that run on Zenrows’ side with no machine of your own, see Run scraping jobs on a schedule.

Read a run’s trace

Every fetch, extract, batch, crawl and browser run writes a record under .zenrows/runs/<run-id>/. The run id is in the status line (run r-1a9dfd1e).

CLI

zenrows trace explain r-1a9dfd1e
zenrows trace inspect r-1a9dfd1e

Trace explain · r-1a9dfd1e
what happened:   zenrows fetch on https://www.scrapingcourse.com/ecommerce/ succeeded (HTTP 200, 6763 bytes).
failure reason:  none
evidence:        status=ok cost=$0.0010 · 1 credit cap=protected_fetch
next action:     Reuse these parameters for similar targets; consider capturing them as a skill.

trace inspect prints the full record as JSON, including the exact request parameters and costCredits. trace replay prints the command to run it again.

Customize it

Change	How
Geo-target a request	`--proxy-country us` on `fetch` or `batch create`, see CLI fetch flags
Pick your own render and proxy settings	`--manual --js-render --premium-proxy`, see CLI fetch
Extract with CSS selectors	`zenrows extract <url> --css '{"title":"h1","price":".price"}'`, see CLI extract
Only the failed tasks again	`zenrows batch retry-failed <job-id>`, see CLI batch
Cap what one run may spend	`zenrows policy set max_credits_per_run 500`, see Track usage and control spend
Check credits left	`zenrows usage`

Troubleshooting

AUTH_MISSING: no key in this workspace. Run zenrows login --api-key YOUR_ZENROWS_API_KEY from the project folder.
No workspace found on trace: traces live in .zenrows/. Run zenrows init --yes in the project folder, then run the command again.
PARAM_CONFLICT_AUTO_MANUAL: --js-render or --premium-proxy on fetch or extract needs --manual. Add it, or drop those flags to keep mode=auto.
POLICY_LIMIT_EXCEEDED on batch create: the estimate is above your max_credits_per_run. Split the file or raise the cap with zenrows policy set max_credits_per_run <n>.
The cron job finds no key: cron starts in your home folder. cd into the project folder first, as in the script above.
The cron job reports zenrows: command not found: cron runs with a minimal PATH. Call the CLI by its full path, which which zenrows prints.

More codes and fixes: CLI troubleshooting.

Scrape a list of URLs

Batch from the API and SDKs, with webhooks and reruns.

Prompts that put Zenrows MCP to work

The same jobs, done by your AI agent.

Track usage and control spend

Read per-request cost and set caps.

Resolve common response codes

Symptom-to-fix for 422, 429, 402 and more.

Start Here

Foundations Course

Fetch Recipes

Extract Recipes

Crawl Recipes

Batch Recipes

Browser Recipes

Pipelines

Use Cases

By Surface

Production

Automate scraping from your terminal with the CLI

Ask your AI agent

Install and log in

Fetch and extract into jq

Script with —json

Run a URL list as one Batch job

Crawl a site, then batch what you found

Drive a browser from the shell

Schedule it with cron

Read a run’s trace

Customize it

Troubleshooting

Scrape a list of URLs

Prompts that put Zenrows MCP to work

Track usage and control spend

Resolve common response codes

​Ask your AI agent

​Install and log in

​Fetch and extract into jq

​Script with —json

​Run a URL list as one Batch job

​Crawl a site, then batch what you found

​Drive a browser from the shell

​Schedule it with cron

​Read a run’s trace

​Customize it

​Troubleshooting

​Related recipes

Scrape a list of URLs

Prompts that put Zenrows MCP to work

Track usage and control spend

Resolve common response codes

Ask your AI agent

Install and log in

Fetch and extract into jq

Script with —json

Run a URL list as one Batch job

Crawl a site, then batch what you found

Drive a browser from the shell

Schedule it with cron

Read a run’s trace

Customize it

Troubleshooting

Related recipes