> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Async Requests and Concurrency with the Python SDK
> Scrape multiple URLs in parallel with the ZenRows Python SDK's built-in concurrency and asyncio support.
When you need to scrape many URLs at once, calling `client.get()` in a loop wastes time waiting for each request to finish before starting the next one. The SDK's async methods let you fire off requests in parallel instead.
## Basic usage
Pass `concurrency` when creating the client to cap how many requests run at the same time:
```python theme={null} theme={null}
from zenrows import ZenRowsClient
client = ZenRowsClient("YOUR_ZENROWS_API_KEY", concurrency=5, retries=1)
```
Set `concurrency` to match your ZenRows plan's concurrent request limit. Each client instance enforces its own limit. Two scripts (or two client instances) don't share it, so running both at once can still trigger `429 Too Many Requests` errors. See [Concurrency](/universal-scraper-api/features/concurrency) for how plan limits work.
Use `get_async` with `asyncio.gather` to run requests in parallel and wait for all of them to finish:
```python theme={null} theme={null}
from zenrows import ZenRowsClient
import asyncio
client = ZenRowsClient("YOUR_ZENROWS_API_KEY", concurrency=5, retries=1)
async def main():
urls = [
"https://www.scrapingcourse.com/ecommerce/",
# ... more URLs
]
responses = await asyncio.gather(*[client.get_async(url) for url in urls])
for response in responses:
print(response.status_code, response.text)
asyncio.run(main())
```
`asyncio.gather` waits for every call to finish, even if some of them fail. It won't raise until all results (or exceptions) are collected. Each item in `responses` is a full `requests.Response`, so you can inspect `status_code`, `text`, and `headers` exactly as you would with `client.get()`.
Always run async code with `asyncio.run(main())`. Calling `main()` directly returns a coroutine object without executing it, and Python will raise `RuntimeWarning: coroutine 'main' was never awaited`.
## When to use async and concurrency
Use `get_async`/`post_async`/`put_async` with `asyncio.gather` when scraping more than a handful of URLs and you want them processed in parallel instead of one at a time. For a single URL, `client.get()` (see [Quickstart](/universal-scraper-api/sdk/python/quickstart)) is simpler and does the same thing without the `asyncio` boilerplate.
## Async POST and PUT
`post_async` and `put_async` work the same way, accepting the same `data`, `params`, and `headers` arguments as their synchronous counterparts:
```python theme={null} theme={null}
responses = await asyncio.gather(*[
client.post_async(url, data={"key1": "value1"}) for url in urls
])
```
```python theme={null} theme={null}
responses = await asyncio.gather(*[
client.put_async(url, data={"key1": "value1"}) for url in urls
])
```
## How it works under the hood
The SDK runs each async call on a thread pool sized to your `concurrency` value, then awaits the results, giving you the ergonomics of `asyncio` without needing an async HTTP client. This means the same retry behavior configured via `retries` applies equally to synchronous and async calls. See [Error Handling and Retries](/universal-scraper-api/sdk/python/error-handling-and-retries) for details.
## Troubleshooting
* **`RuntimeWarning: coroutine 'main' was never awaited`**: You called `main()` directly instead of `asyncio.run(main())`.
* **`429 Too Many Requests` even though `concurrency` is set correctly**: Another script or process is using the same API key at the same time. Concurrency limits are per plan, not per client instance, so simultaneous scripts share the same real ceiling even though each `ZenRowsClient` only tracks its own local limit.
* **Some responses in the list are much slower than others**: This is expected; total throughput depends on the slowest requests in each concurrent batch. See the [Concurrency](/universal-scraper-api/features/concurrency#impact-of-request-duration-on-throughput) guide for the underlying math.
* **Cancelling a request doesn't free a concurrency slot right away**: ZenRows keeps the slot occupied for up to a few minutes after a client-side cancellation while it finishes processing server-side.
## Pricing
Async requests are billed identically to synchronous ones. Cost depends on the `params` you send per request, not on how many run in parallel:
| Configuration | Cost multiplier vs. basic |
| --------------------------------------- | ------------------------- |
| Basic request | 1x |
| `js_render=True` | 5x |
| `premium_proxy=True` | 10x |
| `js_render=True` + `premium_proxy=True` | 25x |
Running requests concurrently doesn't add cost by itself. It only changes how fast the same requests complete. See [ZenRows Pricing](/first-steps/pricing).
## FAQ (Frequently Asked Questions)
The default concurrency is 5. You can set it to any number you want according to your ZenRows plan.
No. The concurrency limit is per `ZenRowsClient` instance. Separate instances or separate scripts don't share limits with each other, even though your ZenRows plan enforces one real limit across all of them.
Yes. `multiprocessing.pool.ThreadPool` with plain `requests` calls works too. See the [Concurrency](/universal-scraper-api/features/concurrency#python-with-requests) guide for that approach if you'd rather not use the SDK's async methods.