Wait For Parameter

The Wait For parameter instructs ZenRows to pause JavaScript rendering until a specific element appears on the page, identified by a CSS selector. Unlike fixed delays, Wait For provides dynamic waiting that adapts to actual page loading conditions, making your scraping more reliable and efficient. When you specify a CSS selector with the wait_for parameter, ZenRows continuously monitors the page during JavaScript rendering until the target element becomes visible in the DOM. This ensures that critical content has loaded before capturing the HTML, regardless of the duration of the loading process.

The Wait For parameter requires js_render=true to function, as it operates within the browser environment during JavaScript rendering.

How the Wait For parameter works

The Wait For parameter actively monitors the page’s DOM structure during JavaScript rendering, checking repeatedly for the presence of your specified CSS selector. Once the target element appears and becomes visible, ZenRows immediately captures the HTML content. This process ensures you capture:

Elements that load at unpredictable times
Content dependent on API response timing
Elements that appear after user interactions
Dynamically generated content with variable loading speeds
Critical page components that indicate full loading completion

The monitoring continues until either the element appears or the maximum timeout of 3 minutes is reached. If the selector is not found within this timeframe, ZenRows returns a 422 error indicating that the element could not be located.

Basic usage

Add the wait_for parameter with a CSS selector to your JavaScript rendering request:

# pip install requests
import requests

url = 'https://www.scrapingcourse.com/ecommerce/'
apikey = 'YOUR_ZENROWS_API_KEY'
params = {
    'url': url,
    'apikey': apikey,
    'js_render': 'true',
    'wait_for': '.price',
}
response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response.text)

This example waits until an element with the class price appears on the page before capturing the HTML content. The waiting time adapts automatically to the actual loading speed. The waiting time automatically adapts to the actual loading speed, up to a maximum of 3 minutes.

CSS selector examples

Use various CSS selectors to target different types of elements:

Python

# Wait for specific classes
params = {'wait_for': '.content-loaded'}

# Wait for IDs
params = {'wait_for': '#main-content'}

# Wait for specific attributes
params = {'wait_for': '[data-loaded="true"]'}

# Wait for nested elements
params = {'wait_for': '.product-container .price-display'}

# Wait for elements with specific text content
params = {'wait_for': '.status:contains("Available")'}

# Wait for form elements
params = {'wait_for': 'input[name="search"]'}

# Wait for multiple class combinations
params = {'wait_for': '.product.loaded.visible'}

When to use the Wait For parameter

The Wait For parameter is essential in these scenarios:

Dynamic content loading:

AJAX-loaded content - Elements populated by asynchronous requests
API-dependent data - Content that appears after external API calls
Progressive loading - Pages that load content in stages
Conditional content - Elements that appear based on user state or preferences
Real-time updates - Content that updates based on live data feeds

User interface elements:

Interactive components - Buttons, forms, and controls that load dynamically
Navigation elements - Menus and links that appear after initialization
Modal dialogs - Pop-ups and overlays that appear programmatically
Data visualizations - Charts, graphs, and tables that render after data loading
Search results - Results that appear after query processing

Troubleshooting

If ZenRows cannot find a matching element for the CSS selector, it will retry internally several times. If it still doesn’t match after the 3-minute timeout, the request will return a 422 error. This means your selector likely does not exist in the final HTML, or is too fragile to be reliable.

Common reasons for the 422 error when using wait_for

Selector Not Present in Final HTML

Inspect the site using browser DevTools

Open the page
Right-click the target content and choose “Inspect”
Check if your selector exists after the page fully loads

Verify your selector

Run document.querySelectorAll('your_selector') in the browser console
If it returns no elements, your selector is incorrect

Tips

Use simple selectors like .class or #id
Prefer stable attributes like [data-testid="item"]
Avoid overly specific or deep descendant selectors

Dynamic or Fragile Selectors

Some websites use auto-generated class names that change frequently. These are considered dynamic and unreliable.

Re-check the page in DevTools if a previously working selector fails.
Look for stable attributes like data-*.
Use attribute-based selectors, which are more stable.

Instead of this:

Python

params = {'wait_for': '.xY7zD1'} # e.g., Google Search
params = {'wait_for': '.product_list__V9tjod'} # A mix of readable and random

Use stable alternatives:

Python

params = {'wait_for': '[data-testid="product-list"]'}
params = {'wait_for': 'img[src$=".jpg"]'}
params = {'wait_for': '[data-products="item"]'}

You can also combine multiple fallback selectors:

Python

params = {'wait_for': '.product, .listing, [data-products="item"]'}

Track your CSS selectors over time. When the target website changes its structure, you’ll likely need to update your selectors.

Content Is Conditional or Missing

When scraping at scale, it’s common to encounter pages where the expected content is missing or appears under certain conditions.

Common Scenarios Where Selectors Might Fail

Out-of-stock products: The product is valid, but some elements like the price or “Add to cart” button are missing.
Deleted or unavailable pages: You may be accessing product URLs directly, but the product has been removed. The site may return a 404 error or a custom error page without updating the URL.
Failed pages: The page might fail to load properly causing your selector to not match any on the HTML.
Conditional rendering: Some content is only rendered based on user location, browser behavior, or scrolling. Especially on JavaScript-heavy websites.

How to Handle It

Use the following ZenRows parameters to help identify these cases:

original_status=true
Returns the original HTTP status from the target site. Helps distinguish between a bad selector and a broken page.
Python
```
params = {
    'original_status': 'true'
}
```
For more details check the Original Status Documentation
allowed_status_codes=404,500
Lets you capture and analyze error pages instead of discarding them.
Python
```
params = {
    'allowed_status_codes': '404,500,503'
}
```
For more details check the Allowed Status Codes Documentation
Best practices:
- Anticipate that some selectors may not match if content is missing or the page structure changes.
- Consider checking for fallback selectors or error indicators (like a 404 message or error class).
- Monitor your scraping jobs for unexpected increases in 422 errors, which may indicate site changes, missing data, or blocking.

The CSS Selector Exists but Still Fails

Sometimes, your CSS selector is correct but still triggers a 422 error. Here are possible causes:

CSS selector is present but hidden (display: none)
ZenRows considers it valid. If you need a visible element, consider using a child or wrapper that only appears when the content is shown.
You can find more information about advanced CSS selectors here.
CSS selector appears after user interaction
Use the js_instructions to simulate a click or scroll action first.
The page relies on external scripts (slow loading)
Try a different wait_for selector that appears earlier in the loading process. Alternatively, switch to our Scraping Browser, which offers longer session times and allows you to manipulate requests more deeply through Puppeteer or Playwright.
CSS selector Typos:
Double-check for spelling errors, missing dots (.) for classes, or missing hashes (#) for IDs.

Alternative: Manual Wait

Instead of waiting for a selector, you can add a fixed delay using the wait parameter:

Python

params = {'wait': 10000,} # Wait of 10 seconds

Useful when dynamic elements take time to appear but don’t have consistent selectors. The maximum wait time is 30 seconds.

Combine with Premium Proxy for Protected Sites

Use Wait For with Premium Proxy for maximum effectiveness:

Python

params = {
    'url': url,
    'apikey': apikey,
	'js_render': 'true',
	'wait_for': '.price',
	'premium_proxy': 'true',
}

Real-World Case Example

Let’s say you’re targeting the product grid on scrapingcourse.com/ecommerce/, and your request fails with a 422 error while your wait_for CSS selector is:

Python

params = {'wait_for': '.lists-product',}

This selector is syntactically valid, but it does not match any element in the HTML of the target page, so ZenRows cannot proceed and returns a 422 error. Inspecting the product section reveals the correct selectors:

HTML

<ul class="products columns-4" id="product-list" data-testid="product-list" data-products="list">
    <li data-products="item" class="product ...">
        <a href="..." class="woocommerce-LoopProduct-link ...">
        <img src="..." class="product-image ..." />
        <h2 class="product-name woocommerce-loop-product__title">Abominable Hoodie</h2>
        <span class="price" data-testid="product-price"><span class="product-price woocommerce-Price-amount amount"><bdi><span class="woocommerce-Price-currencySymbol">$</span>69.00</bdi></span></span>
        </a>
        <a class="button product_type_variable" data-product_id="246" data-product_sku="MH09">Select options</a>
    </li>
    <!-- more li.product ... -->
</ul>

Valid alternatives:

Python

params = {'wait_for': '#product-list'} # Waits for the ID 'product-list'
params = {'wait_for': '.products .product'} # Waits for any product in the product list
params = {'wait_for': '[data-products="item"]'} # Waits for any product item using a data attribute
params = {'wait_for': 'h2.product-name'} # Waits for product names
params = {'wait_for': '[data-testid="product-list"]'} # Waits for the entire product list container

CSS Selector Cheat Sheet

Selector	Example	Use Case
`.class`	`.price`	Wait for an elements with class “price”
`#id`	`#main-content`	Wait for an element with ID “main-content”
`[data-attr]`	`[data-loaded="true"]`	Wait for attribute presence
`[attr^="val"]`	`[id^="item-"]`	Attribute starts with “item-”
`[attr$="val"]`	`[src$=".png"]`	Attribute ends with “.png”
`A > B`	`.list > .item`	Direct child of a parent
`A B`	`.list .item`	Any descendant item inside a parent
`A, B`	`.price, .discount`	Match either `.price` or `.discount`
`:nth-child(n)`	`li:nth-child(2)`	Select the 2nd child (or any Nth) of its parent
`:first-child`	`div:first-child`	First child of a parent element
`:last-child`	`div:last-child`	Last child of a parent element

Pricing

The wait_for parameter doesn’t increase the request cost. You pay the JavaScript Render (5 times the standard price) regardless of the wait value you choose.

Frequently Asked Questions (FAQ)

Can I use XPath instead of CSS selectors?

Yes! Everything in this guide also applies to XPath. ZenRows fully supports XPath in the wait_for parameter. Please ensure that you provide a valid expression.Here’s an example based on the HTML below:

HTML

<ul class="products columns-4" id="product-list" data-testid="product-list">
<li data-products="item" class="product">
    <a href="/product/abominable-hoodie" class="woocommerce-LoopProduct-link">
    <img src="hoodie.jpg" class="product-image" />
    <h2 class="product-name">Abominable Hoodie</h2>
    <span class="price">$69.00</span>
    </a>
</li>
</ul>

Working XPath Examples:

Python

    params = {'wait_for': '//ul[@id="product-list"]/li'} # Waits for any <li> inside the product list.
    params = {'wait_for': '//h2[contains(@class, "product-name")]'} # Waits for a heading with a class containing product-name.
    params = {'wait_for': '//span[@class="price" and contains(text(), "69")]'} # Waits for price elements that include the number 69.

To test XPath in your browser, open DevTools and use the console command: $x('//your/xpath').

Just as with CSS selectors, ensure that your XPath expressions are accurate and match the final rendered HTML. If not found, you’ll still get a 422 error.

What happens if the element I'm waiting for never appears?

How does wait_for differ from wait when both are specified?

What CSS selectors are supported?

How do I know if my selector is too specific or too general?

Does wait_for work with Premium Proxy?

What's the maximum time wait_for will wait?

Quickstart

Get Started

Features

Troubleshooting

How the Wait For parameter works

Basic usage

CSS selector examples

When to use the Wait For parameter

Dynamic content loading:

User interface elements:

Troubleshooting

Common reasons for the 422 error when using wait_for

Selector Not Present in Final HTML

Dynamic or Fragile Selectors

Content Is Conditional or Missing

Common Scenarios Where Selectors Might Fail

How to Handle It

The CSS Selector Exists but Still Fails

Alternative: Manual Wait

Combine with Premium Proxy for Protected Sites

Real-World Case Example

CSS Selector Cheat Sheet

Pricing

Frequently Asked Questions (FAQ)

Quickstart

Get Started

Features

Troubleshooting

​How the Wait For parameter works

​Basic usage

​CSS selector examples

​When to use the Wait For parameter

​Dynamic content loading:

​User interface elements:

​Troubleshooting

​Common reasons for the 422 error when using wait_for

​Selector Not Present in Final HTML

​Dynamic or Fragile Selectors

​Content Is Conditional or Missing

​Common Scenarios Where Selectors Might Fail

​How to Handle It

​The CSS Selector Exists but Still Fails

​Alternative: Manual Wait

​Combine with Premium Proxy for Protected Sites

​Real-World Case Example

​CSS Selector Cheat Sheet

​Pricing

​Frequently Asked Questions (FAQ)

How the Wait For parameter works

Basic usage

CSS selector examples

When to use the Wait For parameter

Dynamic content loading:

User interface elements:

Troubleshooting

Common reasons for the 422 error when using wait_for

Selector Not Present in Final HTML

Dynamic or Fragile Selectors

Content Is Conditional or Missing

Common Scenarios Where Selectors Might Fail

How to Handle It

The CSS Selector Exists but Still Fails

Alternative: Manual Wait

Combine with Premium Proxy for Protected Sites

Real-World Case Example

CSS Selector Cheat Sheet

Pricing

Frequently Asked Questions (FAQ)