Frequently Asked Questions

Does Usage Carry Over Months?

Can I Scrape Using Browser Extensions?

Can I See the XHR / AJAX / Fetch Requests that Browsers Perform?

Does ZenRows Bypass Antibot Solutions?

Does ZenRows Have No-Code Options?

Does ZenRows Integrate with External Tools?

Does ZenRows offer Custom plans?

How Can I Speed Up My Requests?

Optimizing your requests can significantly improve performance and reduce response times. Below are general best practices, followed by specific recommendations for each ZenRows product.

Factors Affecting Request Speed

Concurrency: Sending multiple requests simultaneously can increase throughput
Resource Usage: Features like JavaScript rendering or waiting for specific conditions can impact speed
Response Size (optional): Pages with dynamic content will naturally take longer to load. Consider targeting only the necessary data or using output filters to minimize payload.
Success Rate: The rate of successful requests. If the success rate is low, you may need to increase the number of requests or the concurrency.

While Residential Proxies have no concurrency restrictions, other products have plan-specific limits. Monitor your performance when adjusting these settings.

Monitoring Concurrency Usage

Each response includes headers that help you manage and optimize your concurrency:

Concurrency-Limit: 200
Concurrency-Remaining: 199

These headers help you:

Monitor how many concurrent requests your plan allows
Track how many slots are currently available
Adjust request volume dynamically to avoid hitting limits that may delay or throttle requests

The Concurrency-Remaining header reflects the real-time state of your concurrency usage and is the primary value our system uses to enforce limits. If it reaches zero and more requests are sent, you may receive a 429 Too Many Requests error and your IP can be temporarily blocked for 5 minutes.

If you receive a BLK0001 error (IP Address Blocked), it means your IP has exceeded the allowed error rate. The block will last for 5 minutes and will impact your ability to send new requests during that time, affecting your overall scraping speed. For more details, see our API Error Codes documentation.

Use these headers to adjust your request flow in real-time, scaling up when possible and backing off before hitting limits.

Product-Specific Recommendations

Universal Scraper API

Optimize JavaScript Rendering:

Disable js_render=true for static content to improve speed
Only enable when necessary for dynamic content or accessing protected content
Consider the impact on response times

Minimize Wait Times:

Use wait and wait_for only when required
Set the minimum necessary wait duration
Longer waits mean slower requests

Use Premium Proxies:

Enable premium_proxy=true for faster, more consistent responses
Particularly useful for sites with anti-bot measures
Can reduce retries and improve overall speed

Scraper APIs

Concurrency Management:

Start with moderate concurrency and monitor response times
Increase gradually while maintaining acceptable speed
Implement backoff strategies when requests slow down

Parameter Optimization:

Remove unnecessary parameters that might slow down requests
Only use parameters essential for your use case
Monitor the impact of each parameter on response times

Residential Proxies

Request Rate Optimization:

Monitor response times at different request rates
Adjust based on target site performance
Implement backoff when responses slow down

Scraping Browser

Resource Management:

Disable unnecessary JavaScript execution
Block non-essential resources (images, media, etc.)
Optimize browser settings for faster loading

CAPTCHA Handling:

Implement manual CAPTCHA solving to avoid automated delays
Consider the impact on overall request speed

Speed Optimization Best Practices

Start with Baseline: Begin with standard settings and measure response times
Monitor Performance: Use response headers and timing metrics to track speed
Gradual Optimization: Make incremental changes and measure their impact
Smart Retries: Use exponential backoff for failed requests to maintain speed
Target-Specific Tuning: Adjust settings based on the specific website’s performance

While these optimizations can improve request speed, some features (like JavaScript rendering) might be necessary for your specific use case. If you need help optimizing for speed while maintaining functionality, our support team is here to assist.

How Is Usage Counted?

How to bypass CAPTCHAs?

What is concurrency?

Concurrency refers to the number of ongoing requests that happen at any given time. By different means, computers and languages can call the API in parallel and wait for results while others are still running. You can use concurrency with any ZenRows plan; check out pricing for more details.For more details, check our how-to guide on concurrency to see details about implementation in Python and JavaScript.

Important Behavior to Understand

Canceled Requests Still Count Against Your Concurrency

One crucial thing to understand is that canceling requests on the client side does NOT immediately free up concurrency slots. When you cancel a request:

Your client stops waiting for a response
But the ZenRows server continues processing the request
The concurrency slot remains occupied until the server-side processing completes
Only then is the concurrency slot released

This can lead to unexpected 429 errors if you’re canceling requests and immediately trying to make new ones, as your concurrency limit might still be reached.

Security System for Failing Requests

ZenRows implements a security system that may temporarily ban your API key if you send too many failing requests in a short period. Types of failing requests that can trigger this include:

429 Too Many Requests errors due to exceeding concurrency limits
Invalid API key errors
400 Bad Request errors due to invalid parameters
Other repeated API errors

If your API key gets temporarily banned, you’ll receive an error from the API. If the requests continue, the IP address might get banned for a few minutes and the requests will not even connect with the server.

Monitoring Your Concurrency Usage

You can monitor your concurrency usage through response headers:

Concurrency-Limit: 200
Concurrency-Remaining: 199
X-Request-Cost: 0.001
X-Request-Id: 67fa4e35647515d8ad61bb3ee041e1bb
Zr-Final-Url: https://httpbin.io/anything

These headers provide valuable information about your request:

Concurrency-Limit: Your maximum concurrent request limit based on your plan
Concurrency-Remaining: Number of additional concurrent requests you can make
X-Request-Cost: The cost of this request (varies based on enabled features)
X-Request-Id: Unique identifier for this request - essential when reporting issues to support
Zr-Final-Url: The final URL after any redirects that occurred during the request

How many concurrent requests are included in my plan?

The concurrency limit varies by subscription plan:

Trial plan: 5 concurrent requests
Developer plan: 20 concurrent requests
Startup plan: 50 concurrent requests
Business plan: 100 concurrent requests
Business 500: 150 concurrent requests
Business 1K: 200 concurrent requests
Business 2K: 300 concurrent requests
Business 3K: 400 concurrent requests

Enterprise plans can include custom concurrency limits to fit your needs. Contact us for tailor-made Enterprise solutions.

If get a “429 Too Many Requests” error, do I lose that request or is it queued?

You’ll receive an error, and that request won’t be queued or retried automatically. You’ll need to manage retries on your end, ensuring you don’t exceed your concurrency limit.

Can I increase my concurrency limit?

Absolutely! We offer various plans with different concurrency limits to suit your needs. If you find yourself frequently hitting the concurrency limit, consider upgrading.

How can I monitor my concurrency usage?

When using the Universal Scraper API, each response includes these helpful headers:

Concurrency-Limit: Shows your maximum concurrent request limit
Concurrency-Remaining: Shows how many free concurrency slots you have available

I’ve been blocked by repeatedly exceeding my concurrency limit. Why?

Whenever you exceed your plan concurrency limit, you’ll start receiving “429 Too Many Requests” errors. If you keep sending more and more requests exceeding your plan concurrency limit in a short period of time, the service may temporarily block your IP address to prevent API misuse.The IP address ban will last only a few minutes, but repeatedly being blocked might end in a long-lasting block. Check the concurrency optimization section for more information on how to limit concurrent requests to prevent being blocked.

Troubleshooting with Support

When contacting ZenRows support for any issues with your requests, always include:

The X-Request-Id from the response headers
The exact error message you received
The timestamp of when the error occurred
Details about the parameters you used in the request

This information, especially the Request ID, allows our support team to quickly locate your specific request in our logs and provide more effective assistance.

What is Geolocation and What are All the Premium Proxy Countries?

Many websites tailor their content based on the visitor’s location. For example, Amazon displays different products and prices on its UK (.co.uk) and French (.fr) sites. If you’re scraping data from these sites, using a regional IP ensures that you receive the correct localized content. Also some websites restrict access to their content based on geographic location.To avoid discrepancies caused by regional variations, such as different products being displayed on a retailer’s website, you can send a specific country code with your request. This ensures that your request is localized to the desired country, allowing you to obtain consistent and replicable results.ZenRows supports proxies from numerous countries around the world. You can use any country’s ISO code to configure your proxies.Here is the comprehensive list of premium proxy countries supported by ZenRows:

af => Afghanistan
al => Albania
dz => Algeria
ad => Andorra
ao => Angola
ai => Anguilla
ag => Antigua and Barbuda
ar => Argentina
am => Armenia
au => Australia
at => Austria
az => Azerbaijan
bs => Bahamas
bh => Bahrain
bd => Bangladesh
bb => Barbados
by => Belarus
be => Belgium
bz => Belize
bj => Benin
bm => Bermuda
bt => Bhutan
bo => Bolivia, Plurinational State of
ba => Bosnia and Herzegovina
bw => Botswana
br => Brazil
bn => Brunei Darussalam
bg => Bulgaria
bf => Burkina Faso
bi => Burundi
cv => Cabo Verde
kh => Cambodia
cm => Cameroon
ca => Canada
ky => Cayman Islands
cf => Central African Republic
td => Chad
cl => Chile
cn => China
co => Colombia
km => Comoros
cg => Congo
cd => Congo, The Democratic Republic o
cr => Costa Rica
ci => Cote D'ivoire
hr => Croatia
cu => Cuba
cy => Cyprus
cz => Czech Republic
dk => Denmark
dj => Djibouti
do => Dominican Republic
ec => Ecuador
eg => Egypt
sv => El Salvador
gq => Equatorial Guinea
er => Eritrea
ee => Estonia
et => Ethiopia
fo => Faroe Islands
fj => Fiji
fi => Finland
fr => France
gf => French Guiana
pf => French Polynesia
ga => Gabon
gm => Gambia
ge => Georgia
de => Germany
gh => Ghana
gi => Gibraltar
gr => Greece
gl => Greenland
gp => Guadeloupe
gu => Guam
gt => Guatemala
gn => Guinea
gw => Guinea-Bissau
gy => Guyana
ht => Haiti
hn => Honduras
hk => Hong Kong
hu => Hungary
is => Iceland
in => India
id => Indonesia
ir => Iran, Islamic Republic of
iq => Iraq
ie => Ireland
im => Isle of Man
il => Israel
it => Italy
jm => Jamaica
jp => Japan
jo => Jordan
kz => Kazakhstan
ke => Kenya
kr => Korea, Republic of
kw => Kuwait
kg => Kyrgyzstan
la => Lao People's Democratic Republic
lv => Latvia
lb => Lebanon
ls => Lesotho
lr => Liberia
ly => Libya
lt => Lithuania
lu => Luxembourg
mo => Macao
mk => Macedonia, The Former Yugoslav Republic of
mg => Madagascar
mw => Malawi
my => Malaysia
mv => Maldives
ml => Mali
mt => Malta
mq => Martinique
mr => Mauritania
mu => Mauritius
mx => Mexico
md => Moldova, Republic of
mn => Mongolia
me => Montenegro
ma => Morocco
mz => Mozambique
mm => Myanmar
na => Namibia
np => Nepal
nl => Netherlands
nc => New Caledonia
nz => New Zealand
ni => Nicaragua
ne => Niger
ng => Nigeria
no => Norway
om => Oman
pk => Pakistan
ps => Palestine, State of
pa => Panama
pg => Papua New Guinea
py => Paraguay
pe => Peru
ph => Philippines
pl => Poland
pt => Portugal
pr => Puerto Rico
qa => Qatar
re => Reunion
ro => Romania
ru => Russia
rw => Rwanda
lc => Saint Lucia
mf => Saint Martin (French Part)
ws => Samoa
sa => Saudi Arabia
sn => Senegal
rs => Serbia
sc => Seychelles
sl => Sierra Leone
sg => Singapore
sx => Sint Maarten (Dutch Part)
sk => Slovakia
si => Slovenia
sb => Solomon Islands
so => Somalia
za => South Africa
ss => South Sudan
es => Spain
lk => Sri Lanka
sd => Sudan
sr => Suriname
sz => Swaziland
se => Sweden
ch => Switzerland
sy => Syrian Arab Republic
tw => Taiwan, Province of China
tj => Tajikistan
tz => Tanzania, United Republic of
th => Thailand
tl => Timor-Leste
tg => Togo
to => Tonga
tt => Trinidad and Tobago
tn => Tunisia
tr => Turkey
tm => Turkmenistan
ug => Uganda
ua => Ukraine
ae => United Arab Emirates
gb => United Kingdom
us => United States
uy => Uruguay
uz => Uzbekistan
vu => Vanuatu
ve => Venezuela, Bolivarian Republic of
vn => Viet Nam
ye => Yemen
zm => Zambia
zw => Zimbabwe

How to use it with each product:

Universal Scraper API

Incorporate the selected ISO code into your scraping script to route your requests through the chosen proxy.

scraper.py

# pip install requests
import requests

# Example for setting a proxy for Canada
params = {
    'premium_proxy': 'true',
    'proxy_country': 'ca',
}

response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response.text)

Scraper APIs

If the API offers a country option you can add it similarly to the Universal Scraper API.

# pip install requests
import requests

query_id_url = "example"
api_endpoint = f"https://<INDUSTRY>.api.zenrows.com/v1/targets/<WEBSITE>/<TYPE_OF_REQUEST>/{query_id_url}"

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "country": "us"  # Optional: Target specific country
}

response = requests.get(api_endpoint, params=params)
print(response.text)

Residential Proxies

ZenRows supports IPs from a wide variety of countries, allowing you to access geo-restricted data with ease. You can specify a country by using country followed by the country code in your proxy URL.Example for Spain:

http://<YOUR_USERNAME>:<YOUR_PASSWORD>_country-es@superproxy.zenrows.com:1337

Scraping Browser

You basically have two ways to set a country geolocation on the Scraping Browser, depending if you’re using the SDK or not.Without using the SDK, select a specific country by adding the proxy_country parameter to the WebSocket URL:

wss://browser.zenrows.com?apikey=YOUR_ZENROWS_API_KEY&proxy_country=es

In SDK mode, specify the country when generating the connection URL:

const connectionURL = scrapingBrowser.getConnectURL({
    proxy: {
        location: ProxyCountry.ES
    },
});

By using the right proxy, you can ensure more reliable and geographically relevant data scraping while maintaining compliance with website policies.For further assistance or more detailed configuration, refer to the ZenRows documentation or contact their support team. Happy scraping!

Do you charge failed requests using the Universal Scraper API?

I need to scrape millions of websites every month. Is it possible?

Do I need to install anything to use the product?

Get Started

Help