Custom headers let you tailor your requests by including specific HTTP fields, such as Accept, Cookie, or Referer. These can be useful for:

• Controlling the expected content type (e.g., application/json)
• Maintaining session continuity with cookies
• Mimicking user behavior through referrer URLs

For example, setting the Referer header to https://www.google.com can help simulate a user arriving from a Google search, which some websites use as a trust signal to serve less restricted or personalized content.

​

ZenRows Header Management

ZenRows automatically manages certain browser-related headers to ensure consistency, high success rates, and protection against anti-bot systems. These include headers such as:

• User-Agent
• Accept-Encoding
• Sec-Ch-Ua (Client Hints)
• Sec-Fetch-Mode
• Sec-Fetch-Site
• Sec-Fetch-User

These headers are tightly coupled with browser behavior and are not customizable. Attempts to override them will be ignored by default.

​

Enabling Custom Headers

To include your own headers, set the custom_headers parameter to true in your API request. This enables your custom headers while ZenRows continues to manage sensitive browser-specific ones.

​

Example Use Case: Using Referer

Some websites change their content based on the Referer header. For instance, if you’re scraping a product page and want it to appear as if a user clicked on a Google search result, you might set:

# pip install requests
import requests

url = 'https://httpbin.io/anything'
apikey = 'YOUR_ZENROWS_API_KEY'
params = {
    'url': url,
    'apikey': apikey,
	'custom_headers': 'true',
}
headers = {
	'Referer': 'https://google.com',
}
response = requests.get('https://api.zenrows.com/v1/', params=params, headers=headers)
print(response.text)

This can help bypass redirects, bot checks, or locked content that only appears for certain traffic sources.

If you need to use custom headers and are unsure which ones are allowed or are facing challenges with the headers you’re trying to set, please contact us for guidance.

​

Frequently Asked Questions (FAQ)