Skip to main content
When you need to scrape many URLs at once, calling client.get() in a loop wastes time waiting for each request to finish before starting the next one. The SDK’s async methods let you fire off requests in parallel instead.

​
Basic usage

1

Set a concurrency limit

Pass concurrency when creating the client to cap how many requests run at the same time:
from zenrows import ZenRowsClient

client = ZenRowsClient("YOUR_ZENROWS_API_KEY", concurrency=5, retries=1)
Set concurrency to match your ZenRows plan’s concurrent request limit. Each client instance enforces its own limit. Two scripts (or two client instances) don’t share it, so running both at once can still trigger 429 Too Many Requests errors. See Concurrency for how plan limits work.
2

Scrape a list of URLs concurrently

Use get_async with asyncio.gather to run requests in parallel and wait for all of them to finish:
from zenrows import ZenRowsClient
import asyncio

client = ZenRowsClient("YOUR_ZENROWS_API_KEY", concurrency=5, retries=1)

async def main():
    urls = [
        "https://www.scrapingcourse.com/ecommerce/",
        # ... more URLs
    ]
    responses = await asyncio.gather(*[client.get_async(url) for url in urls])

    for response in responses:
        print(response.status_code, response.text)

asyncio.run(main())
asyncio.gather waits for every call to finish, even if some of them fail. It won’t raise until all results (or exceptions) are collected. Each item in responses is a full requests.Response, so you can inspect status_code, text, and headers exactly as you would with client.get().
3

Run it with asyncio.run

Always run async code with asyncio.run(main()). Calling main() directly returns a coroutine object without executing it, and Python will raise RuntimeWarning: coroutine 'main' was never awaited.

​
When to use async and concurrency

Use get_async/post_async/put_async with asyncio.gather when scraping more than a handful of URLs and you want them processed in parallel instead of one at a time. For a single URL, client.get() (see Quickstart) is simpler and does the same thing without the asyncio boilerplate.

​
Async POST and PUT

post_async and put_async work the same way, accepting the same data, params, and headers arguments as their synchronous counterparts: 
responses = await asyncio.gather(*[
    client.post_async(url, data={"key1": "value1"}) for url in urls
])

​
How it works under the hood

The SDK runs each async call on a thread pool sized to your concurrency value, then awaits the results, giving you the ergonomics of asyncio without needing an async HTTP client. This means the same retry behavior configured via retries applies equally to synchronous and async calls. See Error Handling and Retries for details.

​
Troubleshooting

  • RuntimeWarning: coroutine 'main' was never awaited: You called main() directly instead of asyncio.run(main()).
  • 429 Too Many Requests even though concurrency is set correctly: Another script or process is using the same API key at the same time. Concurrency limits are per plan, not per client instance, so simultaneous scripts share the same real ceiling even though each ZenRowsClient only tracks its own local limit.
  • Some responses in the list are much slower than others: This is expected; total throughput depends on the slowest requests in each concurrent batch. See the Concurrency guide for the underlying math.
  • Cancelling a request doesn’t free a concurrency slot right away: ZenRows keeps the slot occupied for up to a few minutes after a client-side cancellation while it finishes processing server-side.

​
Pricing

Async requests are billed identically to synchronous ones. Cost depends on the params you send per request, not on how many run in parallel:
ConfigurationCost multiplier vs. basic
Basic request1x
js_render=True5x
premium_proxy=True10x
js_render=True + premium_proxy=True25x
Running requests concurrently doesn’t add cost by itself. It only changes how fast the same requests complete. See ZenRows Pricing.

​
FAQ (Frequently Asked Questions)

The default concurrency is 5. You can set it to any number you want according to your ZenRows plan.
No. The concurrency limit is per ZenRowsClient instance. Separate instances or separate scripts don’t share limits with each other, even though your ZenRows plan enforces one real limit across all of them.
Yes. multiprocessing.pool.ThreadPool with plain requests calls works too. See the Concurrency guide for that approach if you’d rather not use the SDK’s async methods.