> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> The Zenrows API (Fetch, Extract via extract=auto, and Batch) is described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Use the spec for exact parameter names and types.

# Node.js SDK

> Start a crawl, wait for it, and read its URLs or HTML with the official Zenrows Node.js SDK. Crawl is included from version 2.2.0.

<Note>
  Crawl is **New**. Any active Zenrows API key can use it. Crawl is on the main `zenrows` package from version 2.2.0, as `client.crawl` on a `ZenRows` instance or as its own `ZenRowsCrawlClient`. Node.js 20 or later is required.
</Note>

Use this guide to start a crawl, wait for it, and read the URLs it keeps, or each page's HTML, with the official Zenrows Node.js SDK. Response fields stay in the API's form (`crawl.crawl_id`, `result.content_url`). The arguments you pass to `create` are camelCase (`maxPages`, `includePatterns`).

<Tip>
  Prefer raw HTTP? [Crawl setup](/crawl/setup) covers the same calls against the REST API. Parameters and errors are in the [endpoint reference](/crawl/endpoints). For the complete method reference, see the [SDK repository on GitHub](https://github.com/ZenRows/zenrows-node-sdk).
</Tip>

## Install and authenticate

```bash theme={"dark"}
npm install "zenrows@^2.2.0"
```

```typescript theme={"dark"}
import { ZenRows } from "zenrows";

const client = new ZenRows("YOUR_ZENROWS_API_KEY");
```

The same client also works on its own:

```typescript theme={"dark"}
import { ZenRowsCrawlClient } from "zenrows";

const crawl = new ZenRowsCrawlClient("YOUR_ZENROWS_API_KEY");
```

Both forms take `{ baseURL, retries, timeout }`. `baseURL` defaults to `https://api.zenrows.com/v1`, `retries` to 3, and `timeout` to 30 seconds per request. Pass them as the second argument of `ZenRowsCrawlClient`, or as `crawl` on the `ZenRows` options (`new ZenRows(apiKey, { crawl: { retries: 3 } })`).

## Quickstart

`create` returns as soon as the crawl is accepted, with `status` `"running"`. `wait` polls until the crawl ends, or until `timeout` seconds pass (default 600). It does not throw on timeout: the crawl comes back still `"running"`. A crawl that ends `"failed"` resolves too, with `error` set. `results` is an async stream that follows `next_cursor` and does not poll, so call `wait` first to read every URL.

`depth` is optional. Leave it out and the crawl follows links until `maxItems` or `maxPages` stops it. Both default to 100 and cap at 100,000.

```typescript theme={"dark"}
const crawl = await client.crawl.create({
  url: "https://www.scrapingcourse.com/ecommerce/",
  depth: 1, // omit for no depth limit
  maxPages: 20, // stop after fetching 20 pages (default 100)
  includePatterns: ["/product/"], // keep only URLs containing one of these
});

const ended = await client.crawl.wait(crawl.crawl_id);
console.log(ended.status, ended.coverage.items_found);

for await (const result of client.crawl.results(crawl.crawl_id)) {
  console.log(result.url);
}
```

<Note>
  `includePatterns` chooses which links Crawl opens, not only which URLs it returns. A homepage crawl with only `/product/` skips listing paths that lack that text, such as `/page/`. See [Crawl setup](/crawl/setup#start-a-crawl).
</Note>

Pass `{ idempotencyKey }` as the second argument of `create` so a retry with the same key and body returns the crawl the first request created.

`get(crawlId, { cursor, limit })` returns one page of results. `limit` is 1 to 10,000 (default 1,000). `next_cursor` is `null` only once the crawl has ended and you have read its last results.

## Read the pages

Set `outputFormat: "html"` to also fetch the page of every kept URL. `content` takes a result, its `content_url`, or its content id. `download` returns the crawl's status (from the `X-Crawl-Status` header) and an async stream of parsed lines.

```typescript theme={"dark"}
const crawl = await client.crawl.create({
  url: "https://www.scrapingcourse.com/ecommerce/",
  maxPages: 20,
  includePatterns: ["/product/"],
  outputFormat: "html",
});
await client.crawl.wait(crawl.crawl_id);

for await (const result of client.crawl.results(crawl.crawl_id)) {
  if (result.content_status === "fetched") {
    const html = await client.crawl.content(crawl.crawl_id, result);
    console.log(result.url, html.length);
  }
}

const download = await client.crawl.download(crawl.crawl_id);
console.log(download.status); // "running" means a later download may hold more lines
for await (const line of download.lines) {
  console.log(line.url);
}
```

On a running crawl, `download` holds only the results kept so far. Call `wait` first for the full file.

With `outputFormat`, each kept page is one more fetch, and `maxPages` counts it. Each page a crawl fetches is one Fetch (formerly, Universal Scraper API) request. See [Crawl pricing](/crawl/endpoints#pricing).

## List and stop

`list` returns one page of the account's crawls, newest first, without their results. `stop` is safe to repeat: a crawl that already ended answers as it ended. Read the final counts with `get`.

```typescript theme={"dark"}
const page = await client.crawl.list({ limit: 20 });
for (const item of page.crawls) {
  console.log(item.crawl_id, item.status);
}

await client.crawl.stop(crawl.crawl_id); // the id from create
```

Pass `page.next_cursor` as `cursor` for the next page. It is absent on the last page.

## Errors

Every non-2xx response throws `ZenRowsCrawlError`, with `status`, `code`, `detail`, and `retryAfter`. A crawl that ended `failed` is not an exception: read `error.code` and `error.detail` on it.

```typescript theme={"dark"}
import { ZenRowsCrawlError } from "zenrows";

try {
  await client.crawl.create({
    url: "https://www.scrapingcourse.com/ecommerce/",
    maxPages: 20,
  });
} catch (err) {
  if (err instanceof ZenRowsCrawlError && err.code === "too_many_crawls") {
    console.log(err.status, err.detail, err.retryAfter);
  } else {
    throw err;
  }
}
```

An account already running 3 active jobs, shared with [Batch](/batch/introduction), gets `429` `too_many_crawls`. `create` does not retry that response. The other codes are in the [endpoint reference](/crawl/endpoints#errors).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.