Click, fill, and scroll before you scrape

Some content appears only after a visitor acts: a “Load more” button, an infinite feed, a search box. Add js_instructions to a Fetch request and Zenrows performs those actions in the browser before it returns the page. Uses: Fetch with js_instructions, js_render and json_response

Ask your AI agent

With the Zenrows MCP server connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.

Use Zenrows to read https://www.scrapingcourse.com/button-click after clicking the
"Load more" button twice. Then tell me how many products are on the page and the names of the last three.

Click a “Load more” button

js_instructions is a JSON array of steps that run in order. Here, two clicks on the button with waits in between load two extra pages of products. css_extractor returns only the product names.

import json
import requests

instructions = [
    {"click": "#load-more-btn"},
    {"wait": 1500},
    {"click": "#load-more-btn"},
    {"wait": 1500},
]

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://www.scrapingcourse.com/button-click",
    "js_render": "true",
    "js_instructions": json.dumps(instructions),
    "css_extractor": json.dumps({"names": ".product-name"}),
}

response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
response.raise_for_status()

names = response.json()["names"]
print(len(names), "products loaded")
print(names[-3:])

const instructions = [
  { click: "#load-more-btn" },
  { wait: 1500 },
  { click: "#load-more-btn" },
  { wait: 1500 },
];

const params = new URLSearchParams({
  apikey: "YOUR_ZENROWS_API_KEY",
  url: "https://www.scrapingcourse.com/button-click",
  js_render: "true",
  js_instructions: JSON.stringify(instructions),
  css_extractor: JSON.stringify({ names: ".product-name" }),
});

const response = await fetch(`https://api.zenrows.com/v1/?${params}`, {
  signal: AbortSignal.timeout(180_000),
});
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

const { names } = await response.json();
console.log(names.length, "products loaded");
console.log(names.slice(-3));

curl -G "https://api.zenrows.com/v1/" \
  --data-urlencode "apikey=YOUR_ZENROWS_API_KEY" \
  --data-urlencode "url=https://www.scrapingcourse.com/button-click" \
  --data-urlencode "js_render=true" \
  --data-urlencode 'js_instructions=[{"click":"#load-more-btn"},{"wait":1500},{"click":"#load-more-btn"},{"wait":1500}]' \
  --data-urlencode 'css_extractor={"names":".product-name"}'

zenrows fetch https://www.scrapingcourse.com/button-click \
  --js-instructions '[{"click":"#load-more-btn"},{"wait":1500},{"click":"#load-more-btn"},{"wait":1500}]' \
  --out products.html

The CLI runs js_instructions in mode=auto, which renders the page and bills it at the JavaScript rendering rate.

Example output

The page starts with 12 products, and each click adds 12 more:

36 products loaded
['Mithra Warmup Pant', 'Thorpe Track Pant', 'Zeppelin Yoga Pant']

Scroll an infinite feed

Use scroll_y with a pixel distance, followed by a wait so the next batch can load. Repeat the pair once per batch you need.

Python (Requests)

import json
import requests

instructions = [
    {"scroll_y": 3000},
    {"wait": 1500},
    {"scroll_y": 3000},
    {"wait": 1500},
    {"scroll_y": 3000},
    {"wait": 1500},
]

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://www.scrapingcourse.com/infinite-scrolling",
    "js_render": "true",
    "js_instructions": json.dumps(instructions),
    "css_extractor": json.dumps({"names": ".product-name"}),
}

response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
response.raise_for_status()

names = response.json()["names"]
print(len(names), "products loaded")

Output: 48 products loaded.

Fill and submit a search field

fill takes a selector and a value. Follow it with a click on the submit button and a wait so the results page loads. Zr-Final-Url tells you where the browser ended up.

Python (Requests)

import json
import requests

instructions = [
    {"fill": ["#wp-block-search__input-1", "hoodie"]},
    {"click": ".wp-block-search__button"},
    {"wait": 3000},
]

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://www.scrapingcourse.com/ecommerce/",
    "js_render": "true",
    "js_instructions": json.dumps(instructions),
    "css_extractor": json.dumps({"results": ".entry-title a"}),
}

response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
response.raise_for_status()

print("Landed on:", response.headers["Zr-Final-Url"])
print(response.json()["results"][:5])

Output:

Landed on: https://www.scrapingcourse.com/ecommerce/?s=hoodie
['Eos V-Neck Hoodie', 'Daphne Full-Zip Hoodie', 'Selene Yoga Hoodie', 'Miko Pullover Hoodie', 'Hera Pullover Hoodie']

When a form submits with a plain GET, like most search boxes, you can also request the results URL directly (/ecommerce/?s=hoodie) and skip the interaction.

Check which steps ran

Add json_response=true and the response becomes JSON with a js_instructions_report: one entry per step, with its success and duration. Use it whenever the result is missing something.

Python (Requests)

import json
import requests

instructions = [
    {"click": "#load-more-btn"},
    {"wait_for": ".product-item:nth-child(13)"},
]

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://www.scrapingcourse.com/button-click",
    "js_render": "true",
    "js_instructions": json.dumps(instructions),
    "json_response": "true",
}

response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
response.raise_for_status()

report = response.json()["js_instructions_report"]
print(json.dumps(report, indent=2))

Output:

{
  "instructions_duration": 1002,
  "instructions_executed": 2,
  "instructions_succeeded": 2,
  "instructions_failed": 0,
  "instructions": [
    {"instruction": "click", "success": true, "params": {"selector": "#load-more-btn"}, "duration": 799},
    {"instruction": "wait_for", "success": true, "params": {"selector": ".product-item:nth-child(13)"}, "duration": 203}
  ]
}

The page HTML is in the same JSON, under html.

Customize it

Change	How
Tick a checkbox or pick a dropdown option	`{"check": "<selector>"}`, `{"select_option": ["<selector>", "<value>"]}`, see Available instructions
Wait for the network to settle	`{"wait_event": "networkidle"}`
Run your own JavaScript	`{"evaluate": "document.querySelector('.cookie-banner')?.remove()"}`
Act inside an iframe	Use the `frame_*` variants, see Working with iframes
Long multi-page flows, logins, downloads	Use Browser, see Fill and submit forms

Troubleshooting

A step shows "success": false in the report: the selector did not match a visible element. Inspect the rendered page and copy the selector again, or add a wait_for on the element before acting on it.
The item count does not grow after a click: the new items had no time to load. Raise the wait after each click, or use wait_for with a selector of the first new item.
A step after a form submit reports failure: the page navigated while the step waited. Use a fixed wait after the submit click, then read the result.
The request takes long or times out on your side: long instruction lists add up. Set your HTTP client timeout to 180 seconds or more and keep waits as short as the page allows.

Scrape JavaScript-rendered pages

Render the page and wait for the element you need.

Capture screenshots, PDFs, and files

Capture the page after your interactions.

Scroll and paginate to collect every item

Drive long feeds step by step from a real browser.

Keep the same IP and submit forms

Post form data straight to the target.

Start Here

Foundations Course

Fetch Recipes

Extract Recipes

Crawl Recipes

Batch Recipes

Browser Recipes

Pipelines

Use Cases

By Surface

Production

Click, fill, and scroll before you scrape

Ask your AI agent

Click a “Load more” button

Example output

Scroll an infinite feed

Fill and submit a search field

Check which steps ran

Customize it

Troubleshooting

Scrape JavaScript-rendered pages

Capture screenshots, PDFs, and files

Scroll and paginate to collect every item

Keep the same IP and submit forms

​Ask your AI agent

​Click a “Load more” button

​Example output

​Scroll an infinite feed

​Fill and submit a search field

​Check which steps ran

​Customize it

​Troubleshooting

​Related recipes

Scrape JavaScript-rendered pages

Capture screenshots, PDFs, and files

Scroll and paginate to collect every item

Keep the same IP and submit forms

Ask your AI agent

Click a “Load more” button

Example output

Scroll an infinite feed

Fill and submit a search field

Check which steps ran

Customize it

Troubleshooting

Related recipes