This guide shows you the most common web scraping scenarios and the exact parameters you need for each one. Use these ready-to-implement recipes to quickly solve your specific scraping challenges with the Universal Scraper API.

​
Use Case Overview

Use CaseKey ParametersWhen You Need This
Load dynamic pages (SPAs, dashboards, infinite scroll)js_render, wait_for, block_resources, js_instructionsWhen content loads via JavaScript or requires user interaction
Keep session/IP across requestssession_id, custom_headersFor multi-step workflows like account access → search → checkout
Submit forms or simulate user actionsjs_instructions, js_render, wait_forWhen you need to click buttons, fill forms, or navigate pages
Access geo-restricted contentpremium_proxy, proxy_countryWhen content varies by location or is region-locked
Handle pagination and infinite scrolljs_render, js_instructions, wait_forWhen content loads dynamically as you scroll or click “Load More”
Extract only specific data fieldscss_extractor, autoparse, outputsWhen you want structured data instead of full HTML
Capture API calls and background requestsjson_response, js_render, wait_forWhen the data you need comes from AJAX/XHR requests
Bypass anti-bot protectionpremium_proxy, js_render, custom_headers, session_idWhen sites use Cloudflare or other bot detection
Debug failed requestsoriginal_status, allowed_status_codes, screenshotWhen requests fail and you need to understand why
Optimize performance and costsblock_resources, outputs, wait_forWhen you want faster responses and lower bandwidth usage

​
Code Recipes

​
1. Search for products using the search form

This example shows how to automatically fill and submit a search form, then wait for the search results to load.
Python
import requests

url = "https://api.zenrows.com/v1/"
params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://www.scrapingcourse.com/ecommerce/",
    "js_render": "true",
    "js_instructions": """[
        {"wait_for":"#wp-block-search__input-1"},
        {"fill":["#wp-block-search__input-1","hoodie"]},
        {"wait":500},
        {"click":"#block-2 > form > div > button"},
        {"wait_for":"#main"}
    ]""",
    "premium_proxy": "true",
}
response = requests.get(url, params=params)
print(response.text)
The js_instructions parameter fills the search field with “hoodie”, clicks the search button, and waits for the main content area to load with search results before returning the content.
Head to the JS Instructions Documentation for more details.

​
2. Maintain a Session Across Requests

Use the same session_id value to maintain cookies and session state across multiple requests.
Python
import requests

# Define session ID and API endpoint
session_id = "1234"
url = "https://api.zenrows.com/v1/"

# First request establishes a session
params1 = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/",
  "session_id": session_id
}
response1 = requests.get(url, params=params1)

# Second request reuses the same session
params2 = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/pagination",
  "session_id": session_id
}
response2 = requests.get(url, params=params2)
This approach is essential for workflows where you need to maintain the same IP across requests.
Learn more about the session_id parameter here.

​
3. Extract Specific Fields Only

Instead of getting the full HTML, extract only the data you need using CSS selectors.
Python
params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/ecommerce/",
  "css_extractor": """{"product_name":".product-name","price":".price"}""",
}
Returns clean, structured data:
JSON
{
  "price": [
    "$69.00",
    "$57.00",
    "$48.00",
    "$24.00",
    "$74.00",
    "$7.00",
    "$45.00",
    "$69.00",
    "$40.00",
    "$42.00",
    "$34.00",
    "$32.50",
    "$20.00",
    "$22.00",
    "$39.00",
    "$45.00"
  ],
  "product_name": [
    "Abominable Hoodie",
    "Adrienne Trek Jacket",
    "Aeon Capri",
    "Aero Daily Fitness Tee",
    "Aether Gym Pant",
    "Affirm Water Bottle",
    "Aim Analog Watch",
    "Ajax Full-Zip Sweatshirt",
    "Ana Running Short",
    "Angel Light Running Short",
    "Antonia Racer Tank",
    "Apollo Running Short",
    "Arcadio Gym Short",
    "Argus All-Weather Tank",
    "Ariel Roll Sleeve Sweatshirt",
    "Artemis Running Short"
  ]
}
Check the CSS Extractor Documentation for more details.

​
4. Scrape Localized Content

Access geo-restricted content or get localized pricing by routing your request through specific countries.
Python
params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/ecommerce/",
  "premium_proxy": "true",
  "proxy_country": "de"
}
This example routes your request through Germany, allowing you to see content as it appears to German users.
Find more details about geolocation on the Proxy Country Documentation.

​
5. Debugging a Blocked Request

When requests fail, use these parameters to understand what’s happening without losing the response data.

​
Check the actual HTTP status

Use the original_status parameter to see the real HTTP status code the target page returns.
Python
params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://example.com/blocked",
  "original_status": "true",
}
response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response)
Check more about the original_status parameter here.

​
Allow specific error status codes

Use the allowed_status_codes parameter to receive the actual HTML content even when the page returns error status codes.
Python
params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://example.com/blocked",
  "allowed_status_codes": "403,404,500",
}
response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response.text)
Find out more on the Allowed Status Codes Documentation.

​
Add custom headers

Use custom headers to simulate browser behavior and avoid detection.
Python
params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://example.com/blocked",
}
headers = {
    'cookie': 'your-cookie',
    'referer': 'https://www.google.com',
}
response = requests.get('https://api.zenrows.com/v1/', params=params, headers=headers)
print(response.text)
ZenRows handles all browser-based headers. To find out more about it, check our Headers Documentation.

​
What’s Next

Ready to implement these recipes? Here are your next steps: