How to Integrate Puppeteer with ZenRows

Puppeteer is the most popular headless browser library in JavaScript, but anti-bot solutions can easily detect and prevent it from accessing web pages. Avoid that limitation with a ZenRows Puppeteer integration!

​

Use ZenRows’ Proxies in Puppeteer to Avoid Blocks

ZenRows offers residential proxies in 190+ countries that auto-rotate the IP address for you and offer Geolocation and http/https protocols. Integrate them into Puppeteer to appear as a different user every time so that your chances of getting blocked are reduced exponentially.

You have three ways to get a proxy with ZenRows, one is via Residential Proxies, where you get our proxy, and it’s charged by the bandwidth; the other way is via the Universal Scraper API’s Premium Proxy, which is our residential proxy for the API, and you are charged by the request, depending on the params you choose; and the third is by using the Scraping Browser where can integrate into your code with just one line of code.

After logging in, you’ll get redirected to the Request Builder page, then go to the Proxies Generator page and create your proxy:

Select your Proxy Username, Proxy Country, Protocol, and Sticky TTL. Finally, copy your Proxy URL or use the cURL example at the bottom of the page.

http://<YOUR_USERNAME>:<YOUR_PASSWORD>@superproxy.zenrows.com:1337

The target site of this tutorial section will be httpbin.io/ip, an endpoint that returns the origin IP of the incoming request. You’ll use it to verify that ZenRows is working.

Consider the following basic Puppeteer script. It launches a headless browser, navigates to a target URL, and extracts the page’s content (in this case, the IP address returned by the API endpoint).

import puppeteer from "puppeteer";

(async () => {
    // Launch Chrome or Chromium in headless mode
    // and open a new blank page
    const browser = await puppeteer.launch();
    const page = await browser.newPage();

    // Visit the target URL
    await page.goto("https://httpbin.io/ip");

    // Retrieve the page content and log it
    const body = await page.waitForSelector("body");
    const response = await body.getProperty("textContent");
    const jsonResponse = await response.jsonValue();
    console.log(jsonResponse);

    await browser.close();
})();

​

Configure your Residential Proxy in Puppeteer

To use authenticated residential proxies with Puppeteer, you’ll need to use Puppeteer’s --proxy-server flag and then page.authenticate with username and password.

const browser = await puppeteer.launch({
  args: [`--proxy-server=superproxy.zenrows.com:1337`],
});
const page = await browser.newPage();
await page.authenticate({username: '<YOUR_USERNAME>', password: '<YOUR_PASSWORD>'});

import puppeteer from "puppeteer";

(async () => {
    // Launch a headless Chromium instance with ZenRows Residential Proxies configured
    const browser = await puppeteer.launch({
        args: [`--proxy-server=superproxy.zenrows.com:1337`],
    });
    const page = await browser.newPage();
    await page.authenticate({username: '<YOUR_USERNAME>', password: '<YOUR_PASSWORD>'});

    // Visit the target URL
    await page.goto("https://httpbin.io/ip");

    // Retrieve the page content and log it
    const body = await page.waitForSelector("body");
    const response = await body.getProperty("textContent");
    const jsonResponse = await response.jsonValue();
    console.log(jsonResponse);

    // Release resources
    await browser.close();
})();

node scraper.js

​

Troubleshooting

​

Chrome Not Found Error

If you encounter the error Error: Could not find Chrome, it indicates that Puppeteer is unable to locate the Chrome or Chromium browser on your system. This issue often arises when the browser is installed in a non-standard location or if you’re using a custom installation of Chrome.

To resolve this issue, explicitly specify the path to the Chrome executable in your Puppeteer configuration. You can do this by setting the executablePath option to the correct path where Chrome is installed on your system.

const browser = await puppeteer.launch({
  executablePath: '/your/path/to/chrome-or-chromium',
  args: [`--proxy-server=superproxy.zenrows.com:1337`],
});

Replace /your/path/to/chrome-or-chromium with the actual path to the Chrome executable on your machine.

​

Invalid SSL Certificate

You might encounter a net::ERR_CERT_AUTHORITY_INVALID error due to SSL certificate issues. Configure Puppeteer to ignore SSL certificate errors by adding the following flag to the launch options to resolve this.

const browser = await puppeteer.launch({
  args: [`--proxy-server=superproxy.zenrows.com:1337`],
  ignoreHTTPSErrors: true
});

This allows Puppeteer to access any HTTPS page without SSL certificate issues.

​

Stopped by Bot Detection with Puppeteer: CAPTCHAs, WAFs, and Beyond

Many websites employ advanced anti-bot measures like CAPTCHAs and Web Application Firewalls (WAFs) to prevent automated scraping. Simply using proxies may not be enough to bypass these protections.

Instead of relying solely on Puppeteer, consider using ZenRows’ Universal Scraper API, which provides:

• JavaScript Rendering and Interaction Simulation: Similar to Puppeteer but optimized with anti-bot bypass capabilities.
• Comprehensive Anti-Bot Toolkit: ZenRows offers advanced tools to overcome complex anti-scraping solutions.

​

Pricing

ZenRows operates on a bandwidth usage model on the Residential Proxies; it is pay-per-success on the Universal Scraper API (that means you only pay for requests that produce the desired result).

To optimize your scraper’s success rate, fully replace Pupetteer with ZenRows. Different pages on the same site may have various levels of protection, but using the parameters recommended above will ensure that you are covered.

ZenRows offers a range of plans, starting at just $69 monthly. For more detailed information, please refer to our pricing page.

​

Frequently Asked Questions (FAQs)