The ZenRows® Scraping API is a versatile tool designed to simplify and enhance the process of extracting data from websites. Whether you’re dealing with static or dynamic content, our API provides a range of features to meet your scraping needs efficiently.

Key Features

  • JavaScript Render: Render JavaScript on web pages using a headless browser to scrape dynamic content that traditional methods might miss.

    • Wait times and selectors: Specify wait times and CSS selectors to ensure elements are fully loaded before scraping.
    • Resource blocking: Block unnecessary resources from loading to speed up scraping.
    • JSON response formatting: Receive responses in JSON format, including XHR or Fetch requests.
    • CSS-based data extraction: Extract data using CSS selectors with our CSS extractor feature.
    • Auto-parsing: Automatically parse and extract relevant data from the scraped HTML.
    • Screenshot capabilities: Capture screenshots of the target page, including full-page and specific element screenshots.
    • Various outputs: Lets you specify which data types to extract from the scraped HTML.
    • JavaScript Instructions: Allowing you to interact with web pages dynamically.
  • Premium Proxies: Leverage a vast network of over 55 million residential IPs across 185+ countries, ensuring a 99.9% uptime for uninterrupted scraping.

    • Geolocation selection: Choose the geographical location of the IP address used for the request (for Premium Proxies).
  • Custom Headers: Add custom HTTP headers to your requests for more control.

  • Session management: Use a session ID to maintain the same IP address across multiple requests for up to 10 minutes.

  • Cloudflare bypass: The API is designed to overcome Cloudflare protection, making it easier to scrape protected websites.

  • Language agnostic: While Python examples are provided, the API works with any programming language that can make HTTP requests.

The Scraping API is particularly useful for extracting data from dynamic websites and handling complex scenarios like JavaScript content and web element interactions, without the need to manage proxies or handle dynamic user interactions manually.

Parameter Overview

Customize your scraping requests using the following parameters:

PARAMETERTYPEDEFAULTDESCRIPTION
apikey requiredstringGet Your Free API KeyYour unique API key for authentication
url requiredstringThe URL of the page you want to scrape
js_renderbooleanfalseEnable JavaScript rendering with a headless browser
js_instructionsstringAllows you to interact with the web page dynamically
custom_headersbooleanfalseInclude custom headers in your request
premium_proxybooleanfalseUse premium proxies to make the request harder to detect
proxy_countrystringGeolocation of the IP used to make the request. Only for Premium Proxies
session_idintegerAssign a session ID to maintain the same IP for multiple requests for up to 10 minutes
original_statusbooleanfalseReturn the original HTTP status code from the target page
allowed_status_codesstringReturns the content of the target page even if it fails with a status code on the provided list
wait_forstringWait for a given CSS Selector to load in the DOM before returning the content
waitinteger0Wait a fixed amount of time before returning the content
block_resourcesstringBlock specific resources from loading
json_responsestringfalseObtain the response in JSON format, including data from XHR or Fetch requests
css_extractorstring (JSON)Define CSS Selectors to extract data from the HTML.
autoparsebooleanfalseUse our auto parser algorithm to automatically extract data.
response_typestringGet the content parsed as Markdown, Plaintext or PDF instead of HTML
screenshotbooleanfalseReturns an above-the-fold screenshot of the target page.
screenshot_fullpagebooleanfalseReturns a full-page screenshot of the target page.
screenshot_selectorstringReturns a screenshot of a specific CSS Selector of the target page.
screenshot_formatstringChoose between png and jpeg formats, with PNG being the default
screenshot_qualityintegerApplicable when using JPEG, this parameter allows you to set the quality from 1 to 100
outputsstringLets you specify which data types to extract from the scraped HTML

Getting started

ZenRows API simplifies web scraping, requiring just two essential components:

  1. API key – Your unique key to authenticate requests.
  2. The encoded URL – The target URL you want to scrape, properly encoded.

Connection modes

ZenRows supports three connection methods:

  • API Mode: Ideal for quick and easy scraping with all the processing handled by ZenRows. Just pass your API key and URL, and get the data back.
  • Proxy Mode: If you prefer more control, connect ZenRows as a proxy to your existing scraping infrastructure. This mode allows you to leverage ZenRows’ proxy network while maintaining flexibility in your scraping logic.
  • SDK: For added convenience, ZenRows provides SDKs for Python and Node.js, making it easier for newcomers to get started.

Response Headers

ZenRows simplifies scraping responses by adding custom headers to the returned data. Headers, including cookies, from the target website are prefixed with Zr- for easy identification.

Additionally, ZenRows will append a Zr-Final-Url header showing the final visited URL after any potential redirects, ensuring you always know the last page accessed.

Here’s an example of the headers returned by ZenRows:

Zr-Content-Encoding: gzip
Zr-Content-Type: text/html
Zr-Cookies: _pxhd=Bq7P4CRaW1B...
Zr-Final-Url: https://www.example.com/
As Zr-Final-Url represents the final URL after all redirects, some anti-bot services, such as Cloudflare, may redirect you after a successful request and add parameters to the URL.

API Key

To access the full functionality of ZenRows’ Scraping API, you’ll need a valid API Key. This unique key acts as your personal identifier, ensuring all requests are securely authorized and linked to your account.

Once you have your API Key, you’re ready to start scraping data from any website. If you haven’t created your key yet, you can easily create your API Key and begin exploring all the capabilities ZenRows has to offer.

Your API Key is essential for every request you make, so keep it secure and avoid sharing it with others. If needed, you can regenerate your key or create multiple keys for different projects.

URL

The URL is the web page you wish to scrape. To ensure proper functionality, the URL needs to be URL-encoded, especially if it contains special characters like spaces, question marks, or ampersands. Encoding transforms these characters into a format that browsers and servers can correctly interpret.

Many HTTP clients and ZenRows’ SDKs handle this encoding automatically, but it’s important to be aware of when working with custom implementations.

Always double-check your URLs for proper encoding. If you’re using ZenRows SDKs or common HTTP libraries like requests in Python or fetch in Node.js, they often handle this for you.