Move an existing scraper to Zenrows

Keep your scraper and change only the layer that gets blocked. An HTTP scraper swaps its request for one call to Fetch. A Puppeteer or Playwright scraper swaps launch() for a connection to Browser. Your parsing, storage and scheduling stay as they are, and you can retire proxy rotation and header tricks. Uses: Fetch with Adaptive Stealth Mode (mode=auto), Browser, css_extractor, Extract, Batch

Ask your AI agent

With the Zenrows MCP server connected to Claude Code, Cursor or another coding agent, open your scraper’s code and paste this prompt:

Migrate this scraper to Zenrows, using the Zenrows docs. Send each request through Zenrows
with Adaptive Stealth Mode and a 3-minute timeout, and read the API key from the
ZENROWS_API_KEY environment variable. Remove custom proxies and User-Agent rotation and keep
the parsing code unchanged. Then use Zenrows to read one of the target URLs and confirm the selectors still match.

Choose your path

Your scraper uses	Move it to	What changes
Requests, httpx, axios, `fetch`	Fetch	The request URL and params. Parsing stays
BeautifulSoup or Cheerio selectors	`css_extractor` or Extract	Optional: Zenrows returns JSON, so the parser goes
Puppeteer or Playwright	Browser	`launch()` becomes `connect()`. Page code stays
Scrapy	Scrapy integration	Requests go through Zenrows via a middleware
Selenium	Selenium integration	Page loads go through Zenrows
A loop over thousands of URLs	Batch	One job instead of your own queue

Replace the HTTP request with Fetch

Before, the scraper sends the request itself, with hand-made headers:

Python (before)

import requests

def get_page_content(url):
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
        "Accept-Language": "en-US,en;q=0.9",
    }
    response = requests.get(url, headers=headers)
    response.raise_for_status()
    return response.text

After, Fetch sends it. mode=auto renders JavaScript and uses premium proxies only when a page needs them, so you drop headers and proxies. The parsing function doesn’t change:

Python (after)

import requests
from bs4 import BeautifulSoup

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

def get_page_content(url):
    params = {
        "apikey": ZENROWS_API_KEY,
        "url": url,
        "mode": "auto",
    }
    response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
    response.raise_for_status()
    return response.text

# Your parsing logic stays the same
def scrape_price_data(url):
    soup = BeautifulSoup(get_page_content(url), "html.parser")
    name = soup.select_one("h1.product_title")
    price = soup.select_one("p.price")
    return {
        "name": name.get_text(strip=True) if name else None,
        "price": price.get_text(strip=True) if price else None,
    }

print(scrape_price_data("https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/"))

Drop the parser with a CSS extractor or Extract

When you’re ready, let Zenrows return JSON instead of HTML. Move your selectors into a css_extractor and delete the parsing code:

Python

import json
import requests

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

def scrape_price_data(url):
    params = {
        "apikey": ZENROWS_API_KEY,
        "url": url,
        "mode": "auto",
        "css_extractor": json.dumps({"name": "h1.product_title", "price": "p.price"}),
    }
    response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
    response.raise_for_status()
    return response.json()

print(scrape_price_data("https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/"))

On retailers Zenrows has prepared, such as amazon.com, walmart.com, ebay.com and bestbuy.com, skip selectors entirely: replace css_extractor with extract=auto and read the product from the parsed field. See Get structured JSON from any page.

Replace the local browser with Zenrows Browser

Puppeteer and Playwright scrapers keep their page code. Only the line that starts the browser changes: connect to wss://browser.zenrows.com instead of launching a local Chromium. Install puppeteer-core or playwright (no bundled browser needed).

from playwright.sync_api import sync_playwright

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

def scrape_with_zenrows_browser(url):
    with sync_playwright() as p:
        # Before: browser = p.chromium.launch()
        browser = p.chromium.connect_over_cdp(f"wss://browser.zenrows.com?apikey={ZENROWS_API_KEY}")
        try:
            page = browser.contexts[0].new_page()
            page.goto(url)
            return page.content()
        finally:
            browser.close()

html = scrape_with_zenrows_browser("https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/")
print(len(html), "<title>" in html)

import puppeteer from "puppeteer-core";

const ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY";

async function scrapeWithZenrowsBrowser(url) {
  // Before: const browser = await puppeteer.launch();
  const browser = await puppeteer.connect({
    browserWSEndpoint: `wss://browser.zenrows.com?apikey=${ZENROWS_API_KEY}`,
  });
  try {
    const page = await browser.newPage();
    await page.goto(url, { waitUntil: "domcontentloaded" });
    return await page.content();
  } finally {
    await browser.close();
  }
}

const html = await scrapeWithZenrowsBrowser("https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/");
console.log(html.length, html.includes("Abominable Hoodie"));

Close the browser in a finally block, as above, so every session ends when your code is done. If your scraper only loads pages and reads HTML, with no clicks or logins, Fetch is the lighter option: it returns the rendered page in one request.

Scale out with Batch

If your scraper loops over a long URL list with its own queue, retries and concurrency control, hand the list to Batch instead: one job, parallel fetching, automatic tracking of failures, and results you download when the job ends. The same mode, css_extractor and extract parameters go in the job’s zenrows_params. See Scrape a list of URLs.

Migration checklist

Before: list every place that sends a request or launches a browser, note the custom headers, cookies and proxies each one uses, and test Zenrows on a few real target URLs.
During: replace the request or launch() call, keep mode=auto as the default, set client timeouts to 180 seconds, and read the API key from an environment variable.
After: remove proxy rotation and User-Agent lists, drop parsing dependencies you replaced with css_extractor or Extract, and log the X-Request-Cost and X-Request-Id response headers.

Example output

The Fetch migration and the css_extractor version return the same data for the test product:

{'name': 'Abominable Hoodie', 'price': '$69.00'}

The Browser migrations return the full rendered page:

122781 True

Customize it

Change	How
Keep your own headers	Add `custom_headers=true` and send the headers with the request, see Headers
Content from one country	Add `proxy_country=us` (any ISO code), see Proxy country
Keep the same IP across requests	Add `session_id` (any integer), see Other features
Wait for an element before returning	Use `js_render=true` with `wait_for=<selector>` instead of `mode=auto`, see Wait for selector
Run more requests at once	Match your workers to the `Concurrency-Limit` response header, see Concurrency
Get markdown for an LLM	Add `response_type=markdown`, see Turn any web page into LLM-ready markdown

Troubleshooting

The selectors return nothing after the migration: Zenrows returned the rendered page, which can differ from the raw HTML your old scraper saw. Save response.text once, check the element’s selector in it, and update the selector.
Requests fail with 429: you’re sending more requests at once than your plan’s concurrency. Cap your workers at the Concurrency-Limit header value.
Requests fail with 402: the account is out of credits (AUTH004) or the API key reached one of its credit caps (AUTH014).
Requests end with a client timeout or 499: your HTTP client gave up first. Set its timeout to 180 seconds.
page.goto hangs in Puppeteer: networkidle0 and networkidle2 wait for every background request. Use waitUntil: "domcontentloaded" and wait for the element you need.

Build a price monitor

A complete monitor with price history and drop alerts.

Get structured JSON from any page

Replace selectors with Extract on prepared retailers.

Connect Playwright or Puppeteer to Zenrows Browser

Every connection option for Browser.

Retry failed requests

Add retries with backoff around your Zenrows calls.

Start Here

Foundations Course

Fetch Recipes

Extract Recipes

Crawl Recipes

Batch Recipes

Browser Recipes

Pipelines

Use Cases

By Surface

Production

Move an existing scraper to Zenrows

Ask your AI agent

Choose your path

Replace the HTTP request with Fetch

Drop the parser with a CSS extractor or Extract

Replace the local browser with Zenrows Browser

Scale out with Batch

Migration checklist

Example output

Customize it

Troubleshooting

Build a price monitor

Get structured JSON from any page

Connect Playwright or Puppeteer to Zenrows Browser

Retry failed requests

​Ask your AI agent

​Choose your path

​Replace the HTTP request with Fetch

​Drop the parser with a CSS extractor or Extract

​Replace the local browser with Zenrows Browser

​Scale out with Batch

​Migration checklist

​Example output

​Customize it

​Troubleshooting

​Related recipes

Build a price monitor

Get structured JSON from any page

Connect Playwright or Puppeteer to Zenrows Browser

Retry failed requests

Ask your AI agent

Choose your path

Replace the HTTP request with Fetch

Drop the parser with a CSS extractor or Extract

Replace the local browser with Zenrows Browser

Scale out with Batch

Migration checklist

Example output

Customize it

Troubleshooting

Related recipes