Integrating ZenRows Scraping Browser with Puppeteer

Discover how to scrape data from any website using ZenRows’ Scraping Browser with Puppeteer. This comprehensive guide demonstrates how to create your first browser automation request capable of handling JavaScript-heavy sites and bypassing sophisticated anti-bot measures.

ZenRows’ Scraping Browser offers cloud-hosted Chrome instances that integrate seamlessly with Puppeteer’s automation framework. From scraping dynamic content to performing complex browser interactions, you can build robust scraping solutions in minutes using Puppeteer’s intuitive API.

​

1. Set Up Your Project

​

Set Up Your Development Environment

Ensure you have the necessary development tools and Puppeteer installed before starting. The Scraping Browser supports both Node.js Puppeteer and Python Pyppeteer implementations.

    # Install Node.js (if not already installed)
    # Visit https://nodejs.org/ or use package managers:
    
    # macOS (using Homebrew)
    brew install node
    
    # Ubuntu/Debian (using NodeSource)
    curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash -
    sudo apt-get install -y nodejs
    
    # Windows (using Chocolatey)
    choco install nodejs

    # Install Puppeteer Core
    npm install puppeteer-core

​

Get Your API Key and Connection URL

Create a Free Account with ZenRows and retrieve your API key from the Scraping Browser Dashboard. This key authenticates your WebSocket connection to our cloud browsers.

​

2. Make Your First Request

Begin with a basic request to familiarize yourself with how Puppeteer connects to the Scraping Browser. We’ll target the E-commerce Challenge page to demonstrate browser connection and title extraction.

// npm install puppeteer-core
const puppeteer = require('puppeteer-core');
// scraping browser connection URL
const connectionURL = 'wss://browser.zenrows.com?apikey=YOUR_ZENROWS_API_KEY';

const scraper = async () => {
    // connect to the scraping browser
    const browser = await puppeteer.connect({ browserWSEndpoint: connectionURL });
    const page = await browser.newPage();
    
    await page.goto('https://www.scrapingcourse.com/ecommerce/');
    console.log(await page.title());
    
    await browser.close();
};

scraper();

Replace YOUR_ZENROWS_API_KEY with your actual API key and execute the script:

node scraper.js

Expected Output:

Your script will display the page title:

ScrapingCourse.com E-commerce Challenge

Excellent! You’ve successfully completed your first web scraping request using ZenRows Scraping Browser with Puppeteer.

​

3. Build a Real-World Scraping Scenario

Now let’s advance to a comprehensive scraping example by extracting product data from the e-commerce site. We’ll enhance our code to collect product names, prices, and URLs using Puppeteer’s robust element selection and data extraction capabilities.

// npm install puppeteer-core
const puppeteer = require('puppeteer-core');

// scraping browser connection URL
const connectionURL = 'wss://browser.zenrows.com?apikey=YOUR_ZENROWS_API_KEY';

const scraper = async (url) => {
    // connect to the scraping browser
    const browser = await puppeteer.connect({
        browserWSEndpoint: connectionURL,
    });
    const page = await browser.newPage();
    
    try {
        await page.goto(url, { waitUntil: 'networkidle2' });
        await page.waitForSelector('.product');
        
        // extract the desired data
        const data = await page.$$eval('.product', (products) =>
            products.map((product) => ({
                name: product.querySelector('.product-name')?.textContent.trim() || '',
                price: product.querySelector('.price')?.textContent.trim() || '',
                productURL: product.querySelector('.woocommerce-LoopProduct-link')?.href || '',
            }))
        );
        
        return data;
    } finally {
        await page.close();
        await browser.close();
    }
};

// execute the scraper function
(async () => {
    const url = 'https://www.scrapingcourse.com/ecommerce/';
    const products = await scraper(url);
    console.log(products);
})();

​

Run Your Application

Launch your script to verify the scraping functionality:

node scraper.js

Example Output:

Your script will collect and display the product information:

[
    {
        "name": "Abominable Hoodie",
        "price": "$69.00",
        "productURL": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/"
    },
    {
        "name": "Artemis Running Short",
        "price": "$45.00",
        "productURL": "https://www.scrapingcourse.com/ecommerce/product/artemis-running-short/"
    }
    // ... additional products
]

Outstanding! 🎉 You’ve successfully implemented a production-ready scraping solution using Puppeteer and the ZenRows Scraping Browser.

​

4. Alternative: Using the ZenRows Browser SDK

For enhanced developer experience, consider using the ZenRows Browser SDK rather than manually managing WebSocket URLs. The SDK streamlines connection handling and offers additional development utilities.

​

Install the SDK

npm install @zenrows/browser-sdk

​

Quick Migration from WebSocket URL

Transitioning from direct WebSocket connections to the SDK requires minimal code changes:

Before (WebSocket URL):

const puppeteer = require('puppeteer-core');
const connectionURL = 'wss://browser.zenrows.com?apikey=YOUR_ZENROWS_API_KEY';

const browser = await puppeteer.connect({ browserWSEndpoint: connectionURL });

After (SDK):

const puppeteer = require('puppeteer-core');
const { ScrapingBrowser } = require('@zenrows/browser-sdk');

const scrapingBrowser = new ScrapingBrowser({ apiKey: 'YOUR_ZENROWS_API_KEY' });
const connectionURL = scrapingBrowser.getConnectURL();
const browser = await puppeteer.connect({ browserWSEndpoint: connectionURL });

​

Complete Example with SDK

// npm install @zenrows/browser-sdk puppeteer-core
const puppeteer = require('puppeteer-core');
const { ScrapingBrowser } = require('@zenrows/browser-sdk');

const scraper = async () => {
    // Initialize SDK
    const scrapingBrowser = new ScrapingBrowser({ apiKey: 'YOUR_ZENROWS_API_KEY' });
    const connectionURL = scrapingBrowser.getConnectURL();
    
    const browser = await puppeteer.connect({ browserWSEndpoint: connectionURL });
    const page = await browser.newPage();
    
    await page.goto('https://www.scrapingcourse.com/ecommerce/');
    console.log(await page.title());
    
    await browser.close();
};

scraper();

​

SDK Benefits

• Streamlined configuration: Eliminates manual WebSocket URL construction
• Enhanced error handling: Provides detailed error messages and debugging capabilities
• Future-proof architecture: Automatic updates to connection protocols and endpoints
• Extended utilities: Access to helper functions and advanced configuration options

​

How Puppeteer with Scraping Browser Helps

Integrating Puppeteer with ZenRows’ Scraping Browser delivers significant advantages for web automation:

​

Key Benefits

• Cloud-hosted browser instances: Execute Puppeteer scripts on remote Chrome browsers, preserving local system resources for other applications.
• Drop-in replacement: Transform existing Puppeteer code to use ZenRows by simply changing the connection method - no architectural changes required.
• Full automation capabilities: Leverage Puppeteer’s complete feature set including form interactions, file handling, network monitoring, and custom JavaScript execution.
• Automatic anti-detection: Benefit from built-in residential proxy rotation and authentic browser fingerprints without additional configuration.
• Proven reliability: Cloud infrastructure delivers consistent performance without the complexity of local browser management.
• Massive scalability: Execute up to 150 concurrent browser instances depending on your subscription plan.
• Network optimization: Reduced latency and improved success rates through globally distributed infrastructure.

​

Troubleshooting

Common challenges when integrating Puppeteer with the Scraping Browser and their solutions:

​

Next Steps

You’ve established a strong foundation for Puppeteer-based web scraping with ZenRows. Continue your journey with these resources:

• Practical Use Cases:
  Explore common automation patterns including screenshot capture, custom JavaScript execution, and form interactions.
• Advanced Automation:
  Master sophisticated workflows and leverage advanced Puppeteer capabilities.
• Complete Scraping Browser Documentation:
  Discover all available features and configuration options for the Scraping Browser platform.
• Puppeteer Web Scraping Guide:
  Explore advanced Puppeteer techniques for complex scraping challenges.
• Pricing and Plans:
  Learn about browser usage calculations and select the optimal plan for your requirements.

​

Frequently Asked Questions (FAQ)