> ## Documentation Index > Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > The Zenrows API (Fetch, Extract via extract=auto, and Batch) is described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Use the spec for exact parameter names and types. # Prepare any website for Extract > Add any website to Zenrows Extract in minutes, then get its pages as clean JSON. Extract learns each website's page shapes once, then returns clean JSON for every page of that kind. To add a new site, prepare it yourself: send one example URL per kind of page, poll the status for a few moments, and start extracting. Every Zenrows account can use the domain once it's ready. **Uses:** the [Extract endpoints](/extract/endpoints) (`GET /v1/extract/domains`, `POST /v1/extract/prepared-domains`, `GET /v1/extract/prepared-domains/{domain}`) and [Extract](/extract/setup) (`extract=auto`) ## Ask your AI agent With the [Zenrows MCP server](/mcp/overview) connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own. ```text theme={"dark"} Use Zenrows to get https://www.allbirds.com/products/womens-tree-runners as structured JSON: the product name, price and every size with its SKU and availability. ``` When a site is new to Extract, the agent still returns JSON. To prepare the domain itself, run the steps below once; the prompt then gets site-tailored fields on every page of that kind, for one product or thousands. ## How it works 1. **Check coverage.** `GET /v1/extract/domains` lists every domain Extract serves now. It's free. 2. **Ask for the domain.** `POST /v1/extract/prepared-domains` with the domain and `example_urls`, one URL per kind of page you want (a product page, a search page). 3. **Poll the status.** `GET /v1/extract/prepared-domains/{domain}` moves through `received`, `preparing` and `verifying` to `ready`. Preparation usually takes seconds to minutes. 4. **Extract.** Send `extract=auto` to any page of the kinds you prepared. The three Extract endpoints are free: none of them fetches a page. ## Check whether a domain is covered ```python Python (Requests) theme={"dark"} import requests response = requests.get( "https://api.zenrows.com/v1/extract/domains", headers={"X-API-Key": "YOUR_ZENROWS_API_KEY"}, timeout=30, ) response.raise_for_status() covered = {entry["domain"] for entry in response.json()["domains"]} print(len(covered), "domains;", "allbirds.com" in covered) ``` ```javascript Node.js theme={"dark"} const response = await fetch("https://api.zenrows.com/v1/extract/domains", { headers: { "X-API-Key": "YOUR_ZENROWS_API_KEY" }, }); if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`); const { domains } = await response.json(); const covered = new Set(domains.map((entry) => entry.domain)); console.log(covered.size, "domains;", covered.has("allbirds.com")); ``` ```bash cURL theme={"dark"} curl "https://api.zenrows.com/v1/extract/domains?apikey=YOUR_ZENROWS_API_KEY" ``` ```text Output (before preparing) theme={"dark"} 423 domains; False ``` Sending `extract=auto` to a new domain returns `403` with [`REQS007`](/api-error-codes#REQS007), free of charge, and the message points to the next step: ```json theme={"dark"} { "code": "REQS007", "detail": "Extract has not been prepared for example-shop.com yet. Ask for it with POST /v1/extract/prepared-domains, then retry once it reports ready. GET /v1/extract/domains lists what Extract serves today.", "status": 403 } ``` ## Prepare the domain and wait until it's ready Send the domain without a scheme, and one example URL per kind of page. Then poll every 30 seconds and branch on `status`: `preparing`, `ready` or `failed`. ```python Python (Requests) theme={"dark"} import time import requests headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"} base = "https://api.zenrows.com/v1/extract/prepared-domains" domain = "allbirds.com" # 1. Ask for the domain, with one example URL per kind of page response = requests.post( base, headers=headers, json={ "domain": domain, "example_urls": ["https://www.allbirds.com/products/mens-tree-runners"], }, timeout=30, ) print(response.status_code, response.json()["state"]) # 2. Poll until it is ready while True: body = requests.get(f"{base}/{domain}", headers=headers, timeout=30).json() print(body["status"], body["state"]) if body["status"] != "preparing": break time.sleep(30) if body["status"] == "failed": print(body["failure"]["code"], body["failure"]["detail"]) ``` ```javascript Node.js theme={"dark"} const headers = { "X-API-Key": "YOUR_ZENROWS_API_KEY", "Content-Type": "application/json" }; const base = "https://api.zenrows.com/v1/extract/prepared-domains"; const domain = "allbirds.com"; // 1. Ask for the domain, with one example URL per kind of page const submit = await fetch(base, { method: "POST", headers, body: JSON.stringify({ domain, example_urls: ["https://www.allbirds.com/products/mens-tree-runners"] }), }); console.log(submit.status, (await submit.json()).state); // 2. Poll until it is ready let body; while (true) { body = await (await fetch(`${base}/${domain}`, { headers })).json(); console.log(body.status, body.state); if (body.status !== "preparing") break; await new Promise((resolve) => setTimeout(resolve, 30_000)); } if (body.status === "failed") console.log(body.failure.code, body.failure.detail); ``` ```bash cURL theme={"dark"} # 1. Ask for the domain curl -X POST "https://api.zenrows.com/v1/extract/prepared-domains?apikey=YOUR_ZENROWS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"domain": "allbirds.com", "example_urls": ["https://www.allbirds.com/products/mens-tree-runners"]}' # 2. Check progress until status is ready curl "https://api.zenrows.com/v1/extract/prepared-domains/allbirds.com?apikey=YOUR_ZENROWS_API_KEY" ``` The statuses from the test run, which went from submission to `ready` in under a minute: ```text Output theme={"dark"} 202 received preparing verifying ready ready ``` The final status record, with `roles` showing each page kind you sent: ```json theme={"dark"} { "domain": "allbirds.com", "status": "ready", "state": "ready", "roles": [ { "role": "detail", "status": "ready", "url": "https://www.allbirds.com/products/mens-tree-runners", "cluster_id": "ea44bc9b0c5f", "detail": "ready; fields the page evidence could not verify were dropped and will return empty: aggregate_rating.rating_value, product.availability" } ], "submitted_at": "2026-10-15T00:49:32.818952Z", "updated_at": "2026-10-15T00:49:37.161497Z" } ``` The `detail` sentence names any field that comes back empty on this site, so you know which fields to read before you build on them. Submitting a domain that's already ready returns `202` with this same record, so a retry is always safe. ## Extract pages from the new domain Once the status reads `ready`, `extract=auto` works on every page of the prepared kind, not just the example URL. This request reads a different product than the one used for preparation: ```python Python (Requests) theme={"dark"} import json import requests params = { "apikey": "YOUR_ZENROWS_API_KEY", "url": "https://www.allbirds.com/products/womens-tree-runners", "mode": "auto", "extract": "auto", } response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180) response.raise_for_status() print(json.dumps(response.json()["parsed"], indent=2)) ``` ```javascript Node.js theme={"dark"} const params = new URLSearchParams({ apikey: "YOUR_ZENROWS_API_KEY", url: "https://www.allbirds.com/products/womens-tree-runners", mode: "auto", extract: "auto", }); const response = await fetch(`https://api.zenrows.com/v1/?${params}`, { signal: AbortSignal.timeout(180_000), }); if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`); const { parsed } = await response.json(); console.log(JSON.stringify(parsed, null, 2)); ``` ```bash cURL theme={"dark"} curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fwww.allbirds.com%2Fproducts%2Fwomens-tree-runners&mode=auto&extract=auto" ``` ```bash CLI theme={"dark"} zenrows extract https://www.allbirds.com/products/womens-tree-runners --out tree-runner.json ``` ## Example output The `parsed` field for the Women's Tree Runner page, trimmed: ```json theme={"dark"} { "product": { "name": "Women's Tree Runner", "price": 100, "currency": "USD", "gender": "womens", "material": "tree", "best_for": ["Traveling", "Walking", "Commuting", "Everyday"], "care_instructions": "Machine Washable", "url": "https://www.allbirds.com/products/womens-tree-runners" }, "materials": [ { "component": "Upper", "material_name": "Tree Knit" }, { "component": "Midsole", "material_name": "SweetFoam®" } ], "technical_specs": { "country_of_origin": "Vietnam", "heel_toe_drop": "9.0mm", "weight": "8.6oz (M9), 7.3oz (W7)" }, "variants": [ { "available": true, "price": 100, "size": "7", "sku": "TR3WMST070" }, { "available": true, "price": 100, "size": "8", "sku": "TR3WMST080" } ] } ``` Fields are tailored to the site: here `materials`, `technical_specs` and per-size `variants` with SKUs. The domain also appears in `GET /v1/extract/domains` now (`424 domains; True`). ## Customize it | Change | How | | - | - | | Search and category pages too | Add a search or category URL to `example_urls`, then use [Turn search and listing pages into JSON](/academy/extract/listing-pages) | | Safe retries of the submission | Send an `Idempotency-Key` header, see [Getting a domain prepared](/extract/endpoints#getting-a-domain-prepared) | | A progress bar | Show `state` and each entry in `roles`, see [Reading the response](/extract/endpoints#reading-the-response) | | Domains your account has submitted | `GET /v1/extract/prepared-domains`, see [the endpoint reference](/extract/endpoints) | | Just a few fields | Add `extract_fields`, see [Get structured JSON from any page](/academy/extract/auto-extraction#keep-just-the-fields-you-need) | | Thousands of pages from the new domain | Put `"extract": "auto"` in a Batch job's `zenrows_params`, see [Scrape a list of URLs](/academy/scrape-from-a-list-of-urls) | ## Troubleshooting * **`failed` with `unpreparable` and a role detail of "the target could not be fetched"**: that example URL returned no usable page. Resubmit with a different URL of the same kind, or with the URLs that worked. * **A role reads "this page was prepared, but the domain was not finalized"**: another URL in the same submission failed. Resubmit without that URL. * **`failed` with `discovery_failed`**: resubmit with `example_urls`, so Zenrows prepares the exact page kinds you point at. * **`409` with `in_progress`**: a preparation for that domain is already running. Poll its status instead of resubmitting. * **`REQS007` on a page of a ready domain**: that page kind needs preparing too. Contact support with the URL to add it. * **A field is always empty**: check the `detail` sentence on the role. It names the fields that site's pages leave empty. See [When a preparation fails](/extract/endpoints#when-a-preparation-fails) for every failure code and the next step for each. ## Related recipes How extract=auto works, with extract\_fields and the SDKs. Collect every item across search and category pages. Type and check every row before you store it. Run Extract on thousands of pages as one Batch job. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.