Skip to main content
Custom headers let you tailor your requests by including specific HTTP fields, such as Accept, Cookie, or Referer. These can be useful for:
  • Controlling the expected content type (e.g., application/json)
  • Maintaining session continuity with cookies
  • Mimicking user behavior through referrer URLs
For example, setting the Referer header to https://www.google.com can help simulate a user arriving from a Google search, which some websites use as a trust signal to serve less restricted or personalized content.

​
ZenRows Header Management

ZenRows automatically manages certain browser-related headers to ensure consistency, high success rates, and protection against anti-bot systems. These include headers such as:
  • User-Agent
  • Accept-Encoding
  • Sec-Ch-Ua (Client Hints)
  • Sec-Fetch-Mode
  • Sec-Fetch-Site
  • Sec-Fetch-User
These headers are tightly coupled with browser behavior and are not customizable. Attempts to override them will be ignored by default.

​
Enabling Custom Headers

To include your own headers, set the custom_headers parameter to true in your API request. This enables your custom headers while ZenRows continues to manage sensitive browser-specific ones.

​
Example Use Case: Using Referer

Some websites change their content based on the Referer header. For instance, if you’re scraping a product page and want it to appear as if a user clicked on a Google search result, you might set: 
# pip install requests
import requests

url = 'https://httpbin.io/anything'
apikey = 'YOUR_ZENROWS_API_KEY'
params = {
    'url': url,
    'apikey': apikey,
	'custom_headers': 'true',
}
headers = {
	'Referer': 'https://google.com',
}
response = requests.get('https://api.zenrows.com/v1/', params=params, headers=headers)
print(response.text)
This can help bypass redirects, bot checks, or locked content that only appears for certain traffic sources. If you need to use custom headers and are unsure which ones are allowed or are facing challenges with the headers you’re trying to set, please contact us for guidance.

​
Frequently Asked Questions (FAQ)

Custom headers allow you to modify the HTTP headers in your requests, such as Referer, Accept, or Cookie, to control how the request is perceived by the target server. This can be useful for handling sessions, mimicking specific browser behaviors, or requesting specific content types.
ZenRows manages certain headers related to browser environment and request consistency, such as User-Agent, Accept-Encoding, Sec-Fetch-*, and Client-Hints. This helps ensure success and reliability when scraping protected or complex websites.
Headers such as Sec-Ch-Ua and Accept-Encoding are tightly coupled with browser behavior and could trigger anti-scraping mechanisms if set improperly. ZenRows prevents manual customization of these headers to ensure optimal success rates and avoid unnecessary blocks.
If you attempt to set a forbidden header, it will be ignored by ZenRows, and the request will proceed with the default browser headers managed by the system. This prevents potential issues that could arise from incorrect configurations.
No, you cannot manually set or override headers like Sec-Ch-Ua, Accept-Encoding, and Sec-Fetch-*. These are managed by ZenRows to optimize performance and prevent blocks during web scraping.