The wait_for parameter in ZenRows tells the scraper to pause until a specific CSS selector is found in the rendered HTML. This feature requires js_render to be set to true. If ZenRows cannot find a matching element for the CSS selector, it will retry several times internally. If it still doesn’t match, the request will return a 422 error. This means your selector likely does not exist in the final HTML, or is too fragile to be reliable. This guide helps you debug wait_for failures and avoid common pitfalls.

​
Understanding wait_for

The wait_for parameter targets dynamic content that loads asynchronously. This includes content that appears after an AJAX call (a background request that updates part of a page without reloading) or JavaScript rendering. Here’s a typical usage:
Python
params = {
    'url': url,
    'apikey': apikey,
    'js_render': 'true',
    'wait_for': '.slow_selector',
}

​
Common Resons for the 422 error when using wait_for

​
Selector Not Present in Final HTML

1

Inspect the site using browser DevTools

  1. Open the page
  2. Right-click the target content and choose “Inspect”
  3. Check if your selector exists after the page fully loads
2

Verify your selector

  1. Run document.querySelectorAll('your_selector') in the browser console
  2. If it returns no elements, your selector is incorrect
ScrapingCourse DevTools Selector Debug
3

Tips

  1. Use simple selectors like .class or #id
  2. Prefer stable attributes like [data-testid="item"]
  3. Avoid overly specific or deep descendant selectors

​
Dynamic or Fragile Selectors

Some websites use auto-generated class names that change frequently. These are considered dynamic and unreliable.
  • Re-check the page in DevTools if a previously working selector fails.
  • Look for stable attributes like data-*.
  • Use attribute-based selectors, which are more stable.
Instead of this:
Python
params = {'wait_for': '.xY7zD1'} # e.g., Google Search
params = {'wait_for': '.product_list__V9tjod'} # A mix of readable and random
Use stable alternatives:
Python
params = {'wait_for': '[data-testid="product-list"]'}
params = {'wait_for': 'img[src$=".jpg"]'}
params = {'wait_for': '[data-products="item"]'}
You can also combine multiple fallback selectors:
Python
params = {'wait_for': '.product, .listing, [data-products="item"]'}
Track your CSS selectors over time. When the target website changes its structure, you’ll likely need to update your selectors.

​
Content Is Conditional or Missing

When scraping at scale, it’s common to encounter pages where the expected content is missing or appears under certain conditions.

​
Common Scenarios Where Selectors Might Fail

  • Out-of-stock products: The product is valid, but some elements like the price or “Add to cart” button are missing.
  • Deleted or unavailable pages: You may be accessing product URLs directly, but the product has been removed. The site might return a 404 error or a custom error page without clearly changing the URL.
  • Failed pages: The page might fail to load properly causing your selector to not match any on the HTML.
  • Conditional rendering: Some content is only rendered based on user location, browser behavior, or scrolling. Especially on JavaScript-heavy websites.

​
How to Handle It

Use the following ZenRows parameters to help identify these cases:
  1. original_status=true
    Returns the original HTTP status from the target site. Helps distinguish between a bad selector and a broken page.
    Python
    params = {
    'original_status': 'true'
}
    For more details check the Original Status Documentation
  2. allowed_status_codes=404,500
    Lets you capture and analyze error pages instead of discarding them.
    Python
    params = {
    'allowed_status_codes': '404,500,503'
}
    For more details check the Allowed Status Codes Documentation
  3. Best practices:
    • Anticipate that some selectors may not match if content is missing or the page structure changes.
    • Consider checking for fallback selectors or error indicators (like a 404 message or error class).
    • Monitor your scraping jobs for unexpected increases in 422 errors, which may indicate site changes, missing data, or blocking.

​
The CSS Selector Exists but Still Fails

Sometimes, your CSS selector is correct but still triggers a 422 error. Here are possible causes:
  • CSS selector is present but hidden (display: none)
    ZenRows considers it valid. If you need a visible element, try a child or a wrapper that only appears when content is shown.
    You can find more information about advanced CSS selectors here.
  • CSS selector appears after user interaction
    Use the js_instructions to simulate a click or scroll action first.
  • The page relies on external scripts (slow loading)
    Try a different wait_for selector that appears earlier in the loading process. Alternatively, switch to our Scraping Browser, which offers longer session times and allows you to manipulate requests more deeply through Puppeteer or Playwright.
  • CSS selector Typos:
    Double-check for spelling errors, missing dots (.) for classes, or missing hashes (#) for IDs.

​
Alternative: Manual Wait

Instead of waiting for a selector, you can add a fixed delay using the wait parameter:
Python
params = {'wait': 10000,} # Wait of 10 seconds
Useful when dynamic elements take time to appear but don’t have consistent selectors. The maximum wait time is 30 seconds.

​
Real-World Case Example

Let’s say you’re targeting the product grid on scrapingcourse.com/ecommerce/, and your request fails with a 422 error while your wait_for CSS selector is:
Python
params = {'wait_for': '.lists-product',}
This selector is syntactically valid, but it does not match any element in the HTML of the target page, so ZenRows cannot proceed and returns a 422 error. Inspecting the product section reveals the correct selectors:
HTML
<ul class="products columns-4" id="product-list" data-testid="product-list" data-products="list">
    <li data-products="item" class="product ...">
        <a href="..." class="woocommerce-LoopProduct-link ...">
        <img src="..." class="product-image ..." />
        <h2 class="product-name woocommerce-loop-product__title">Abominable Hoodie</h2>
        <span class="price" data-testid="product-price"><span class="product-price woocommerce-Price-amount amount"><bdi><span class="woocommerce-Price-currencySymbol">$</span>69.00</bdi></span></span>
        </a>
        <a class="button product_type_variable" data-product_id="246" data-product_sku="MH09">Select options</a>
    </li>
    <!-- more li.product ... -->
</ul>
Valid alternatives:
Python
params = {'wait_for': '#product-list'} # Waits for the ID 'product-list'
params = {'wait_for': '.products .product'} # Waits for any product in the product list
params = {'wait_for': '[data-products="item"]'} # Waits for any product item using a data attribute
params = {'wait_for': 'h2.product-name'} # Waits for product names
params = {'wait_for': '[data-testid="product-list"]'} # Waits for the entire product list container

​
CSS Selector Cheat Sheet

SelectorExampleUse Case
.class.priceWait for an elements with class “price”
#id#main-contentWait for an element with ID “main-content”
[data-attr][data-loaded="true"]Wait for attribute presence
[attr^="val"][id^="item-"]Attribute starts with “item-”
[attr$="val"][src$=".png"]Attribute ends with “.png”
A > B.list > .itemDirect child of a parent
A B.list .itemAny descendant item inside a parent
A, B.price, .discountMatch either .price or .discount
:nth-child(n)li:nth-child(2)Select the 2nd child (or any Nth) of its parent
:first-childdiv:first-childFirst child of a parent element
:last-childdiv:last-childLast child of a parent element

​
Frequently Asked Questions (FAQ)