Retry failed requests

At scale, some Fetch requests meet a transient condition: a page that loads slowly on one attempt, a busy moment, a dropped connection. Retry those with growing waits and they go through. Requests that don’t succeed aren’t charged, so a retry costs the attempt that succeeds. Uses: Fetch with Adaptive Stealth Mode (mode=auto), the API error codes and the SDKs’ Python retries and Node.js retries

Ask your AI agent

With the Zenrows MCP server connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.

Use Zenrows to read each of these URLs. If a request fails with a temporary error,
wait a few seconds and retry it, up to 3 times. If it fails because of the request itself,
don't retry: tell me the error code and what to fix.
https://www.scrapingcourse.com/ecommerce/
https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/

Which errors to retry

Retry the errors that can succeed on the next attempt. Fix the ones that return the same answer every time.

Status	Example codes	What to do
`422`	RESP001	Retry. On a fixed configuration, switch to `mode=auto` so Zenrows escalates
`429`	AUTH006, AUTH008	Retry with backoff, and lower your concurrency to `Concurrency-Limit`
`500`, `502`, `503`, `504`	RESP009, CTX0002	Retry with backoff and jitter
Network error or timeout		Retry. Keep the client timeout at 180 seconds or more
`400`	REQS002, REQS004	Fix the parameters. For `REQS002`, use `mode=auto`
`401`	AUTH001, AUTH002	Fix the API key
`402`	AUTH004, AUTH014	Top up or raise the credit cap
`403`, `404`	AUTH009, RESP002	Fix the account or the target URL

Every error body is JSON with a code field: log it with the URL so you can group failures by cause.

Retry with backoff in your own code

This function retries the retryable statuses with waits of 1, 2 and 4 seconds plus random jitter, switches a fixed configuration to mode=auto after a 422, and stops at once on errors that need a fix.

import random
import time

import requests

API_KEY = "YOUR_ZENROWS_API_KEY"
RETRY_STATUSES = {422, 429, 500, 502, 503, 504}

def fetch_with_retries(url, params=None, max_attempts=4):
    params = {"apikey": API_KEY, "url": url, **(params or {})}

    for attempt in range(1, max_attempts + 1):
        try:
            response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
        except (requests.ConnectionError, requests.Timeout) as error:
            print(f"Attempt {attempt}: network error ({type(error).__name__})")
        else:
            if response.ok:
                return response
            if response.status_code not in RETRY_STATUSES:
                # 400, 401, 402, 403, 404: retrying returns the same answer. Fix the request.
                raise RuntimeError(f"{response.status_code}: {response.text}")
            try:
                code = response.json().get("code")
            except ValueError:  # a body that isn't JSON, such as a proxy error page
                code = None
            print(f"Attempt {attempt}: {response.status_code} {code}")

            # A 422 on a fixed configuration: let Adaptive Stealth Mode pick one instead.
            if response.status_code == 422 and "mode" not in params:
                params.pop("js_render", None)
                params.pop("premium_proxy", None)
                params["mode"] = "auto"

        if attempt < max_attempts:
            delay = min(2 ** (attempt - 1), 30) + random.uniform(0, 1)  # 1s, 2s, 4s ... plus jitter
            time.sleep(delay)

    raise RuntimeError(f"Gave up on {url} after {max_attempts} attempts")

response = fetch_with_retries("https://www.scrapingcourse.com/ecommerce/")
print(response.status_code, len(response.text), "bytes")

const API_KEY = "YOUR_ZENROWS_API_KEY";
const RETRY_STATUSES = new Set([422, 429, 500, 502, 503, 504]);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function fetchWithRetries(url, extraParams = {}, maxAttempts = 4) {
  const params = { apikey: API_KEY, url, ...extraParams };

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      const response = await fetch(`https://api.zenrows.com/v1/?${new URLSearchParams(params)}`, {
        signal: AbortSignal.timeout(180_000),
      });
      if (response.ok) return response;

      const body = await response.text();
      if (!RETRY_STATUSES.has(response.status)) {
        // 400, 401, 402, 403, 404: retrying returns the same answer. Fix the request.
        throw Object.assign(new Error(`${response.status}: ${body}`), { final: true });
      }
      let code;
      try {
        code = JSON.parse(body).code;
      } catch {
        code = undefined; // a body that isn't JSON, such as a proxy error page
      }
      console.log(`Attempt ${attempt}: ${response.status} ${code}`);

      // A 422 on a fixed configuration: let Adaptive Stealth Mode pick one instead.
      if (response.status === 422 && !params.mode) {
        delete params.js_render;
        delete params.premium_proxy;
        params.mode = "auto";
      }
    } catch (error) {
      if (error.final) throw error;
      console.log(`Attempt ${attempt}: network error (${error.name})`);
    }

    if (attempt < maxAttempts) {
      const delay = Math.min(2 ** (attempt - 1), 30) * 1000 + Math.random() * 1000; // 1s, 2s, 4s ... plus jitter
      await sleep(delay);
    }
  }
  throw new Error(`Gave up on ${url} after ${maxAttempts} attempts`);
}

const response = await fetchWithRetries("https://www.scrapingcourse.com/ecommerce/");
console.log(response.status, (await response.text()).length, "bytes");

Example output

A page that loads on the first attempt returns at once:

200 82776 bytes

A target that keeps failing, here https://httpbin.io/status/503, comes back from Zenrows as 422 with code RESP001 on every attempt. The function retries it with growing waits, then reports it:

Attempt 1: 422 RESP001
Attempt 2: 422 RESP001
Attempt 3: 422 RESP001
Attempt 4: 422 RESP001
Gave up on https://httpbin.io/status/503 after 4 attempts

None of those attempts is charged.

Retry with a Requests session

If you already use a requests.Session, mount urllib3’s Retry on it and every call through the session retries on its own. backoff_jitter needs urllib3 2.x, which current requests installs.

Python (Requests)

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retries = Retry(
    total=3,                     # up to 3 retries after the first attempt
    backoff_factor=1,            # waits 0s, 2s, 4s between attempts
    backoff_jitter=1,            # plus up to 1s of random jitter
    status_forcelist=[422, 429, 500, 502, 503, 504],
    respect_retry_after_header=True,
    raise_on_status=False,       # hand back the last response instead of raising
)

session = requests.Session()
session.mount("https://", HTTPAdapter(max_retries=retries))

params = {"apikey": "YOUR_ZENROWS_API_KEY", "url": "https://www.scrapingcourse.com/ecommerce/"}
response = session.get("https://api.zenrows.com/v1/", params=params, timeout=180)
print(response.status_code)

This retries with the same parameters each time. Use the function above when you also want the switch to mode=auto on 422.

Turn on the SDKs’ built-in retries

Pass retries to the client and it retries for you with exponential backoff. Install with pip install zenrows or npm install zenrows.

from zenrows import ZenRowsClient

client = ZenRowsClient("YOUR_ZENROWS_API_KEY", retries=3)

response = client.fetch("https://www.scrapingcourse.com/ecommerce/", params={"mode": "auto"}, timeout=180)
print(response.status_code, response.headers["X-Request-Credits"], "credit")

import { ZenRows } from "zenrows";

const client = new ZenRows("YOUR_ZENROWS_API_KEY", { retries: 3 });

const response = await client.fetch("https://www.scrapingcourse.com/ecommerce/", { mode: "auto" });
console.log(response.status, response.headers.get("X-Request-Credits"), "credit");

Output: 200 1 credit. When every attempt fails, the client returns the last response, so check its status as usual.

SDK	Retries on
Python `ZenRowsClient(retries=N)`	`422`, `429`, `500`, `502`, `503`, `504`
Node.js `new ZenRows(key, { retries: N })`	`422`, `503`, `504` and network errors

Extract’s 422 with code no_data_extracted is a final answer, so both SDKs return it at once. Combine retries with concurrency to run many requests safely, see Run many requests concurrently.

Customize it

Change	How
More or fewer attempts	Change `max_attempts`, `total` or `retries`
Longer waits for busy targets	Raise the base delay or the 30-second cap
Retry a whole list that failed	Collect failed URLs and run them again, or let Batch rerun failed tasks
Fewer `429` errors	Size your pool to `Concurrency-Limit`, see Run many requests concurrently
Look up any error code	See API error codes

Troubleshooting

Every retry returns the same 422: the configuration is too light for that site. Switch to mode=auto, or add js_render=true and premium_proxy=true. When content loads late, add wait_for.
429 keeps coming back during retries: retries add to the requests in flight. Lower your pool size and keep the backoff.
A 400 comes back without a retry: the request itself needs a fix. Read the code and see API error codes.
499 or timeouts on your side: your client gave up before Zenrows finished. Set the timeout to 180 seconds or more before adding retries.

Resolve common response codes

The fix for each error code you are likely to see.

Run many requests concurrently

Size your pool to your plan and keep 429s away.

Scrape pages protected by anti-bot systems

Let Adaptive Stealth Mode pick the right configuration per page.

Scrape a list of URLs

Let Batch queue, retry and store results for you.

Start Here

Foundations Course

Fetch Recipes

Extract Recipes

Crawl Recipes

Batch Recipes

Browser Recipes

Pipelines

Use Cases

By Surface

Production

Retry failed requests

Ask your AI agent

Which errors to retry

Retry with backoff in your own code

Example output

Retry with a Requests session

Turn on the SDKs’ built-in retries

Customize it

Troubleshooting

Resolve common response codes

Run many requests concurrently

Scrape pages protected by anti-bot systems

Scrape a list of URLs

​Ask your AI agent

​Which errors to retry

​Retry with backoff in your own code

​Example output

​Retry with a Requests session

​Turn on the SDKs’ built-in retries

​Customize it

​Troubleshooting

​Related recipes

Resolve common response codes

Run many requests concurrently

Scrape pages protected by anti-bot systems

Scrape a list of URLs

Ask your AI agent

Which errors to retry

Retry with backoff in your own code

Example output

Retry with a Requests session

Turn on the SDKs’ built-in retries

Customize it

Troubleshooting

Related recipes