> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> The Zenrows API (Fetch, Extract via extract=auto, and Batch) is described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Use the spec for exact parameter names and types.

# Automate scraping from your terminal with the CLI

> Fetch, extract, crawl, run batch jobs and drive a browser from your terminal with the Zenrows CLI, and schedule it all with cron.

The `zenrows` CLI wraps every Zenrows primitive in one command, writes clean output to stdout and progress to stderr, and keeps a trace of every run. That makes it a natural fit for shell scripts, `jq` pipelines and cron jobs.

**Uses:** [`zenrows fetch`](/cli/fetch/introduction), [`zenrows extract`](/cli/extract/introduction), [`zenrows batch`](/cli/batch/introduction), [`zenrows crawl`](/cli/crawl/introduction), [`zenrows browser`](/cli/browser/introduction), [traces](/cli/troubleshooting)

## Ask your AI agent

With the [Zenrows MCP server](/mcp/overview) connected, or the CLI available to a coding agent such as Claude Code, paste this prompt:

```text theme={"dark"}
Use Zenrows to find the product pages on https://www.scrapingcourse.com/ecommerce/ (up to 10 page loads),
scrape them all in one batch as markdown, and save each page to a folder called pages/.
```

## Install and log in

Install the CLI once, then set up a workspace with your API key. `zenrows init` creates the local `.zenrows/` folder that holds your config, policy and run traces.

```bash CLI theme={"dark"}
npm install -g @zenrows/cli
zenrows init --yes --api-key YOUR_ZENROWS_API_KEY
```

In CI or containers, export the key instead and tell the CLI to read it:

```bash CLI theme={"dark"}
export ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY
zenrows login --env
```

## Fetch and extract into jq

`zenrows fetch` and `zenrows extract` use Adaptive Stealth Mode (`mode=auto`) by default. The page body goes to stdout and the status line goes to stderr, so pipes stay clean.

`zenrows extract` returns structured JSON, so you can pipe it straight into `jq`. With `--css`, you name the fields and their selectors:

```bash CLI theme={"dark"}
zenrows extract https://www.scrapingcourse.com/ecommerce/ \
  --css '{"name":".product-name","price":".product .price"}' \
  | jq '[.name, .price] | transpose | map({name: .[0], price: .[1]}) | .[:3]'
```

```json theme={"dark"}
[
  { "name": "Abominable Hoodie", "price": "$69.00" },
  { "name": "Adrienne Trek Jacket", "price": "$57.00" },
  { "name": "Aeon Capri", "price": "$48.00" }
]
```

Without `--css`, `zenrows extract` sends `extract=auto` for site-tailored fields on domains Zenrows has prepared, such as major retailers, and uses Autoparse on other domains.

Save any page as markdown with `--output markdown` and `--out`:

```bash CLI theme={"dark"}
zenrows fetch https://www.scrapingcourse.com/ecommerce/ --output markdown --out catalog.md
```

```text theme={"dark"}
→ Protected Fetch https://www.scrapingcourse.com/ecommerce/ (auto)…
✓ Wrote 6763 bytes → catalog.md
✓ HTTP 200 · 6763 bytes · cost $0.0010 · 1 credit · run r-1a9dfd1e
```

## Script with --json

Every command accepts `--json` and prints a structured result instead of the body or the status line. Use it to branch on status or log cost in a script:

```bash CLI theme={"dark"}
zenrows fetch https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/ --json \
  | jq '{ok, httpStatus, costCredits, runId}'
```

```json theme={"dark"}
{
  "ok": true,
  "httpStatus": 200,
  "costCredits": 1,
  "runId": "r-bcdfc63b"
}
```

## Run a URL list as one Batch job

For tens to thousands of URLs, submit them as one [Batch](/batch/introduction) job instead of a loop of fetches. The CLI reads a JSONL file with one `{"url": ...}` object per line.

<Steps>
  <Step title="Build the job file from a plain URL list">
    ```bash CLI theme={"dark"}
    jq -Rc '{url: .}' urls.txt > jobs.jsonl
    ```
  </Step>

  <Step title="Estimate the cost locally">
    `estimate` validates the file and prints an upper bound, with no cloud call and no credits spent. Tasks in Adaptive Stealth Mode count at their highest possible cost.

    ```bash CLI theme={"dark"}
    zenrows batch estimate jobs.jsonl
    ```

    ```text theme={"dark"}
    Job spec: jobs.jsonl
    valid jobs: 3/3
    estimated credits, upper bound (1x basic / 5x js / 10x proxy / 25x both or auto): 75
    ✓ Spec is valid.
    ```
  </Step>

  <Step title="Submit and wait for the run">
    `--follow` polls until the run finishes. `--output markdown` sets the format of every result.

    ```bash CLI theme={"dark"}
    zenrows batch create jobs.jsonl --output markdown --follow
    ```

    ```text theme={"dark"}
    → Submitting batch job (3 tasks, ~75 credits)…
    ✓ Submitted job 01M4Y9ACBYZZFWWBY7JA8KE6QQ · status: completed
      3/3 completed · 3 successful · 0 failed
    ```
  </Step>

  <Step title="Download every result">
    ```bash CLI theme={"dark"}
    zenrows batch results 01M4Y9ACBYZZFWWBY7JA8KE6QQ --download pages
    ```

    `pages/` gets one file per task plus `_manifest.jsonl`, which maps each file to its URL and status:

    ```json theme={"dark"}
    {"task_id":"01M4Y9ACBYS5SHV7V54EGYVW2D","url":"https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/","status":"successful","type":"markdown","file":"01M4Y9ACBYS5SHV7V54EGYVW2D.md"}
    ```
  </Step>
</Steps>

The run above billed 3 credits in total: the estimate is a ceiling, and `mode=auto` bills only the configuration that succeeds. Check the real spend with `zenrows batch status <job-id> --json`, which returns `stats.spend.credits`.

## Crawl a site, then batch what you found

`zenrows crawl create` discovers URLs from one start page. Feed the results straight into a Batch job:

```bash CLI theme={"dark"}
zenrows crawl create https://www.scrapingcourse.com/ecommerce/ \
  --max-pages 10 --include-patterns /product/ --include-patterns /page/ --follow

zenrows crawl results <crawl-id> --out crawl.jsonl
jq -c 'select(.url | contains("/product/")) | {url}' crawl.jsonl > jobs.jsonl
```

```text theme={"dark"}
→ Starting crawl of https://www.scrapingcourse.com/ecommerce/ (no depth limit)…
✓ Crawl c_01M4Y9B3GA71076Z7S3MZ2XGP7 finished · status: completed (max_items)
  100 item(s) found · 10 page(s) fetched · 0 failed
```

Including `/page/` lets the crawl walk the paginated listing, and the `jq` filter keeps only product URLs (88 of the 100 in this run). See [Crawl from a seed URL](/academy/scrape-and-crawl-from-a-seed-url) for more patterns.

## Drive a browser from the shell

When a page needs clicks or typing, open a Browser session. Each verb takes the `--session` id that `open` returns. Always close the session when you finish.

```bash CLI theme={"dark"}
SESSION=$(zenrows browser open https://www.scrapingcourse.com/button-click --json | jq -r .session_id)

zenrows browser click --session "$SESSION" --selector "#load-more-btn"
zenrows browser wait --session "$SESSION" --ms 2000
zenrows browser query-selector-all --session "$SESSION" --selector ".product-name" --json \
  | jq -r '.result.elements[].text'

zenrows browser close --session "$SESSION"
```

```text theme={"dark"}
Chaz Kangeroo Hoodie
Teton Pullover Hoodie
Bruno Compete Hoodie
Frankie Sweatshirt
```

For longer flows, put the steps in a JSON file and run them with `zenrows browser run script.json`, which closes the session for you. See the [Browser CLI guide](/cli/browser/introduction).

## Schedule it with cron

The CLI keeps your key and policy in `.zenrows/`, so a cron job only needs to run from that folder. This script refreshes the product pages every morning:

```bash scrape.sh theme={"dark"}
#!/usr/bin/env bash
set -euo pipefail
cd /home/me/catalog

JOB=$(zenrows batch create jobs.jsonl --output markdown --follow --json | jq -r .jobId)
zenrows batch results "$JOB" --download "pages/$(date +%F)"
```

```bash crontab theme={"dark"}
0 6 * * * /home/me/catalog/scrape.sh >> /home/me/catalog/scrape.log 2>&1
```

For schedules that run on Zenrows' side with no machine of your own, see [Run scraping jobs on a schedule](/academy/batch/scheduled-jobs).

## Read a run's trace

Every `fetch`, `extract`, `batch`, `crawl` and `browser` run writes a record under `.zenrows/runs/<run-id>/`. The run id is in the status line (`run r-1a9dfd1e`).

```bash CLI theme={"dark"}
zenrows trace explain r-1a9dfd1e
zenrows trace inspect r-1a9dfd1e
```

```text theme={"dark"}
Trace explain · r-1a9dfd1e
what happened:   zenrows fetch on https://www.scrapingcourse.com/ecommerce/ succeeded (HTTP 200, 6763 bytes).
failure reason:  none
evidence:        status=ok cost=$0.0010 · 1 credit cap=protected_fetch
next action:     Reuse these parameters for similar targets; consider capturing them as a skill.
```

`trace inspect` prints the full record as JSON, including the exact request parameters and `costCredits`. `trace replay` prints the command to run it again.

## Customize it

| Change | How |
| - | - |
| Geo-target a request | `--proxy-country us` on `fetch` or `batch create`, see [CLI fetch flags](/cli/fetch/introduction#flags) |
| Pick your own render and proxy settings | `--manual --js-render --premium-proxy`, see [CLI fetch](/cli/fetch/introduction) |
| Extract with CSS selectors | `zenrows extract <url> --css '{"title":"h1","price":".price"}'`, see [CLI extract](/cli/extract/introduction) |
| Only the failed tasks again | `zenrows batch retry-failed <job-id>`, see [CLI batch](/cli/batch/introduction) |
| Cap what one run may spend | `zenrows policy set max_credits_per_run 500`, see [Track usage and control spend](/academy/production/control-costs) |
| Check credits left | `zenrows usage` |

## Troubleshooting

* **`AUTH_MISSING`**: no key in this workspace. Run `zenrows login --api-key YOUR_ZENROWS_API_KEY` from the project folder.
* **`No workspace found` on `trace`**: traces live in `.zenrows/`. Run `zenrows init --yes` in the project folder, then run the command again.
* **`PARAM_CONFLICT_AUTO_MANUAL`**: `--js-render` or `--premium-proxy` on `fetch` or `extract` needs `--manual`. Add it, or drop those flags to keep `mode=auto`.
* **`POLICY_LIMIT_EXCEEDED` on `batch create`**: the estimate is above your `max_credits_per_run`. Split the file or raise the cap with `zenrows policy set max_credits_per_run <n>`.
* **The cron job finds no key**: cron starts in your home folder. `cd` into the project folder first, as in the script above.
* **The cron job reports `zenrows: command not found`**: cron runs with a minimal `PATH`. Call the CLI by its full path, which `which zenrows` prints.

More codes and fixes: [CLI troubleshooting](/cli/troubleshooting).

## Related recipes

<CardGroup cols={2}>
  <Card title="Scrape a list of URLs" icon="list" href="/academy/scrape-from-a-list-of-urls">
    Batch from the API and SDKs, with webhooks and reruns.
  </Card>

  <Card title="Prompts that put Zenrows MCP to work" icon="message-chatbot" href="/academy/surfaces/mcp-prompts">
    The same jobs, done by your AI agent.
  </Card>

  <Card title="Track usage and control spend" icon="coin" href="/academy/production/control-costs">
    Read per-request cost and set caps.
  </Card>

  <Card title="Resolve common response codes" icon="tool" href="/academy/production/fix-common-errors">
    Symptom-to-fix for 422, 429, 402 and more.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.