Lesson 2: Turn a product page into JSON

In Lesson 1 you fetched the catalog as HTML and markdown. Now you stop parsing pages yourself and let Zenrows return the data as JSON. What you’ll build: two scripts. catalog.py lists every product on a catalog page as JSON with Extract. product_page.py reads the exact fields you choose from one product page with CSS selectors, the same fields the tracker collects for every product in Lesson 4. What you’ll learn:

How extract=auto turns a page into structured JSON with no selectors.
How extract_fields trims the JSON to the fields you need.
How css_extractor returns exactly the elements you point at, as JSON.
How the official Python SDK wraps the request you wrote by hand in Lesson 1.

Time: about 10 minutes.

Install the Python SDK

The fetch() helper from Lesson 1 is a thin wrapper over https://api.zenrows.com/v1/. The official SDK ships the same wrapper as ZenRowsClient, plus clients for Crawl and Batch that you’ll use in Lessons 3 and 4. Install it in your project folder:

pip install zenrows

client.fetch(url, params={...}) sends a Fetch request with your parameters. client.extract(url) sends the same request with extract=auto and mode=auto already set.

Get the catalog as JSON with Extract

Extract reads a page and returns its data in a parsed object, next to the raw html. On the catalog page, parsed holds a products list. Create catalog.py:

Python (SDK)

import json
from zenrows import ZenRowsClient

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"
CATALOG_URL = "https://www.scrapingcourse.com/ecommerce/"

client = ZenRowsClient(ZENROWS_API_KEY)

response = client.extract(CATALOG_URL)
response.raise_for_status()
parsed = response.json()["parsed"]

print("Groups in parsed:", list(parsed))
print("Products found:", len(parsed["products"]))
print(json.dumps(parsed["products"][0], indent=2))

Run python catalog.py:

Groups in parsed: ['page_context', 'products']
Products found: 16
{
  "availability": "in_stock",
  "currency": "USD",
  "image_url": "https://www.scrapingcourse.com/ecommerce/wp-content/uploads/2024/03/mh09-blue_main.jpg",
  "name": "Abominable Hoodie",
  "price": 69,
  "product_id": "246",
  "product_type": "variable",
  "product_url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/",
  "sku": "MH09"
}

You wrote no selectors. Extract recognized a product listing and returned every product with a numeric price, a currency and a link.

Pick the fields you need

parsed carries every field Extract found. Add extract_fields with a comma-separated list, dot-notated inside a group, to keep the ones the tracker uses. Replace the request and the prints in catalog.py:

Python (SDK)

from zenrows import ZenRowsClient

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"
CATALOG_URL = "https://www.scrapingcourse.com/ecommerce/"

client = ZenRowsClient(ZENROWS_API_KEY)

response = client.extract(
    CATALOG_URL,
    params={"extract_fields": "products.name,products.price,products.product_url"},
)
response.raise_for_status()

for product in response.json()["parsed"]["products"][:5]:
    print(f"{product['name']:<26} ${product['price']:>3}  {product['product_url']}")

The output is now a compact price list:

Abominable Hoodie          $ 69  https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/
Adrienne Trek Jacket       $ 57  https://www.scrapingcourse.com/ecommerce/product/adrienne-trek-jacket/
Aeon Capri                 $ 48  https://www.scrapingcourse.com/ecommerce/product/aeon-capri/
Aero Daily Fitness Tee     $ 24  https://www.scrapingcourse.com/ecommerce/product/aero-daily-fitness-tee/
Aether Gym Pant            $ 74  https://www.scrapingcourse.com/ecommerce/product/aether-gym-pant/

Field names come from each site. Send one request without extract_fields first, read the names in parsed, then pin the ones you want. See Selecting the fields you need.

Read exact product fields with CSS selectors

For the tracker you also want details that live on each product page: the SKU, the category, and the sizes and colors a shopper can pick. When you know the page layout, css_extractor is the precise tool: you map a name to a CSS selector, and Zenrows returns those elements as JSON. Add @attribute after a selector to read an attribute instead of the text. Create product_page.py:

Python (SDK)

import json
from zenrows import ZenRowsClient

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"
PRODUCT_URL = "https://www.scrapingcourse.com/ecommerce/product/hero-hoodie/"

PRODUCT_SELECTORS = {
    "name": "h1.product_title",
    "price": "p.price",
    "sku": "span.sku",
    "category": "span.posted_in a",
    "sizes": "select#size option:not(:first-child) @value",
    "colors": "select#color option:not(:first-child) @value",
}

client = ZenRowsClient(ZENROWS_API_KEY)

response = client.fetch(
    PRODUCT_URL,
    params={"mode": "auto", "css_extractor": json.dumps(PRODUCT_SELECTORS)},
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))

Run python product_page.py:

{
  "category": "Hoodies & Sweatshirts",
  "colors": ["Black", "Gray", "Green"],
  "name": "Hero Hoodie",
  "price": "$54.00",
  "sizes": ["XS", "S", "M", "L", "XL"],
  "sku": "MH07"
}

A selector that matches one element returns a string, and one that matches several returns a list. option:not(:first-child) skips the “Choose an option” placeholder in each dropdown. Keep PRODUCT_SELECTORS as it is: Lesson 4 sends it with every product URL.

Extract or CSS selectors? Reach for extract=auto when you want the data a page offers with no setup. Reach for css_extractor when you know the page and want exact elements under your own field names. Both return JSON from the same Fetch request.

Ask your AI agent

With the Zenrows MCP server connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.

Use Zenrows to get the products on https://www.scrapingcourse.com/ecommerce/ as JSON with
name, price and product URL. Then read the SKU, category, sizes and colors from
https://www.scrapingcourse.com/ecommerce/product/hero-hoodie/.

What you built

Your project now has catalog.py and product_page.py. You can:

Get a whole catalog page as JSON with extract=auto, and trim it with extract_fields.
Read exact fields from a known layout with css_extractor, as strings or lists.
Use ZenRowsClient from the official SDK in place of a hand-written request.

You have the data for 16 products on one page. Next, you find the URL of every product in the store.

Next lesson

Lesson 3: Discover every product with Crawl

Start from the catalog and let Crawl follow the links to every product page.

Start Here

Foundations Course

Fetch Recipes

Extract Recipes

Crawl Recipes

Batch Recipes

Browser Recipes

Pipelines

Use Cases

By Surface

Production

Lesson 2: Turn a product page into JSON

Install the Python SDK

Get the catalog as JSON with Extract

Pick the fields you need

Read exact product fields with CSS selectors

Ask your AI agent

What you built

Next lesson

Lesson 3: Discover every product with Crawl

​Install the Python SDK

​Get the catalog as JSON with Extract

​Pick the fields you need

​Read exact product fields with CSS selectors

​Ask your AI agent

​What you built

​Next lesson

Lesson 3: Discover every product with Crawl

Install the Python SDK

Get the catalog as JSON with Extract

Pick the fields you need

Read exact product fields with CSS selectors

Ask your AI agent

What you built

Next lesson