original_status
parameter in the Original HTTP Code documentation.allowed_status_codes
parameter accepts a comma-separated list of HTTP status codes that should be processed and returned as successful responses.
allowed_status_codes
parameter in the Return content on error documentation.wait_for
parameter is critical when scraping dynamic content, but it can be tricky to use correctly, especially with edge cases like empty results. You can specify multiple CSS selectors using standard CSS syntax:
concurrency-remaining
response headercss_extractor
json_response
option can increase payload size, especially when internal API calls are captured. Disable it if not needed.js_render: true
to mimic real browser behavior.premium_proxy: true
to route traffic through residential IPs.'proxy_country': 'us'
or another region where the website is accessible.wait_for
or wait
parameters to ensure dynamic content has fully rendered, especially on slower sites.referer: 'https://www.google.com/'
to mimic typical browser behavior.js_render
, switching proxy regions, or using premium_proxy: true
typically resolves this.x-request-id
response header