Lesson 6: Run the tracker every day

In Lesson 4 you scraped every product once. A tracker earns its name when it runs on its own. In this last lesson you hand the job to Zenrows as a daily schedule, collect each day’s results as a dated snapshot, and learn how to get notified when a run finishes. What you’ll build:

schedule_tracker.py creates a scheduled Batch job that scrapes every product at 09:00 each day.
run_now.py fires the job immediately so you can test it.
save_snapshot.py writes the latest run to products-YYYY-MM-DD.json.
stop_tracker.py pauses and deletes the schedule.
Optionally, webhook_receiver.py gets a signed run.completed event after every run.

What you’ll learn:

How a scheduled job stores its URLs as a template and creates a new run on every fire.
How to set a calendar schedule with a timezone.
How to verify a signed webhook from Zenrows.

Time: about 10 minutes.

Create the daily schedule

A scheduled Batch job keeps its tasks and zenrows_params as a template. Each time the schedule fires, Zenrows starts a fresh run of those tasks, the same kind of run you waited on in Lesson 4: one Fetch request per URL. You set the cadence with one of three builders from zenrows.batch: Rate (every 60 minutes or more, in minute, hour or day units), Calendar (times of day on a daily, weekly or monthly cadence, in a timezone) or At (one future time). Create schedule_tracker.py. It reuses product_urls.txt from Lesson 3 and PRODUCT_SELECTORS from Lesson 2:

Python (SDK)

import json
from zenrows import ZenRowsBatchClient
from zenrows.batch import Calendar, Daily

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

PRODUCT_SELECTORS = {
    "name": "h1.product_title",
    "price": "p.price",
    "sku": "span.sku",
    "category": "span.posted_in a",
    "sizes": "select#size option:not(:first-child) @value",
    "colors": "select#color option:not(:first-child) @value",
}

with open("product_urls.txt") as f:
    product_urls = [line.strip() for line in f if line.strip()]

client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY)

job = client.submit_scheduled(
    Calendar(times_of_day=["09:00"], cadence=Daily(), timezone="Europe/Madrid"),
    [{"url": url, "external_id": url.rstrip("/").split("/")[-1]} for url in product_urls],
    zenrows_params={"mode": "auto", "css_extractor": json.dumps(PRODUCT_SELECTORS)},
    name="product-catalog-tracker-daily",
)

with open("tracker_job_id.txt", "w") as f:
    f.write(job.job_id)

details = job.load().data
print("Scheduled job:", job.job_id)
print("Schedule state:", details.schedule_state.value)
print("Next run at:", details.next_scheduled_run)

Run python schedule_tracker.py:

Scheduled job: 01M4YA87MZSHGRM3SW5093VKNQ
Schedule state: active
Next run at: 2026-10-15 07:00:00+00:00

The next run is 09:00 Madrid time, shown in UTC. The script saves the job ID to tracker_job_id.txt so the next scripts can find the job. Set timezone to any IANA timezone name, and times_of_day to full hours in 24-hour format, such as "09:00" or "18:00".

Test it with a run now

You don’t have to wait until tomorrow. rerun() on a scheduled job that hasn’t fired yet starts a run from its template right away. Create run_now.py:

Python (SDK)

from zenrows import ZenRowsBatchClient

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

with open("tracker_job_id.txt") as f:
    job_id = f.read().strip()

client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY)

run = client.job(job_id).rerun()
print("Fired run", run.run_id)

run = run.wait(timeout=900)
print(f"Run {run.data.status.value}: {run.stats.successful}/{run.stats.total} succeeded")

client.job(job_id) points at the job by ID without fetching it first. Run python run_now.py:

Fired run 01M4YA893ZADACHR1KKMDS11VR
Run completed: 42/42 succeeded

Save a dated snapshot

Each run’s results stay available after it finishes. save_snapshot.py reads the job’s latest run and writes it to a file named after today’s date, so every day adds one snapshot you can compare with the day before:

Python (SDK)

import json
from datetime import date
from zenrows import ZenRowsBatchClient

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

def as_list(value):
    if not value:
        return []
    return value if isinstance(value, list) else [value]

with open("tracker_job_id.txt") as f:
    job_id = f.read().strip()

client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY)
run = client.job(job_id).run.load()
print("Latest run:", run.run_id, run.data.status.value)

products = []
for result in run.download_to_memory(status="successful"):
    fields = json.loads(result.body)
    products.append({
        "slug": result.external_id,
        "name": fields.get("name"),
        "price": fields.get("price"),
        "sku": fields.get("sku"),
        "category": fields.get("category"),
        "sizes": ", ".join(as_list(fields.get("sizes"))),
        "colors": ", ".join(as_list(fields.get("colors"))),
    })

products.sort(key=lambda p: p["slug"])
filename = f"products-{date.today().isoformat()}.json"
with open(filename, "w") as f:
    json.dump(products, f, indent=2)

print(f"Saved {len(products)} products to {filename}")

Run python save_snapshot.py:

Latest run: 01M4YA893ZADACHR1KKMDS11VR completed
Saved 42 products to products-2026-10-15.json

Run save_snapshot.py after each daily run, by hand, from a cron job, or from the webhook in the next section.

Get notified when a run finishes

A webhook tells your app the moment a run ends, so it can save the snapshot without polling. Zenrows POSTs a run.completed event when a run completes and a run.failed event when it fails. With signature: True, each delivery is signed with your organization’s HMAC key, so your receiver can prove it came from Zenrows.

Create a signing key and attach the webhook

Your receiver needs a public HTTPS URL. This script creates your organization’s HMAC key, prints its secret (Zenrows returns it once, so store it now) and attaches the webhook to the tracker job:

Python (SDK)

from zenrows import ZenRowsBatchClient

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

with open("tracker_job_id.txt") as f:
    job_id = f.read().strip()

client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY)

key = client.rotate_hmac_key()
print("Key id:", key.kid)
print("Secret:", key.secret)

client.put_job_webhook(job_id, {"url": "https://your-app.com/webhooks/zenrows", "signature": True})
print("Webhook attached to", job_id)

If your organization already has a key, reuse its secret and skip rotate_hmac_key(). See Manage HMAC keys.

Verify each delivery

Each signed delivery carries an X-Signature header in the form t=<unix>,v1=<hex>,kid=<key id>. Recompute HMAC-SHA256(secret, "<t>.<raw body>") with the base64-decoded secret and compare it to v1. Deliveries arrive at least once, so skip any X-ZenRows-Event-Id you’ve already handled. This Flask receiver (pip install flask) does both:

Python (Flask)

import base64
import hashlib
import hmac

from flask import Flask, abort, request

# Key id -> the base64 secret Zenrows returned when you created the HMAC key.
SECRETS = {"YOUR_HMAC_KEY_ID": "YOUR_HMAC_SECRET"}
seen_event_ids = set()

app = Flask(__name__)

def is_signed_by_zenrows(raw_body, signature_header):
    parts = dict(item.split("=", 1) for item in signature_header.split(",") if "=" in item)
    secret_b64 = SECRETS.get(parts.get("kid"))
    if not secret_b64 or "t" not in parts:
        return False
    secret = base64.b64decode(secret_b64)
    expected = hmac.new(secret, f"{parts['t']}.".encode() + raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, parts.get("v1", ""))

@app.post("/webhooks/zenrows")
def zenrows_webhook():
    if not is_signed_by_zenrows(request.get_data(), request.headers.get("X-Signature", "")):
        abort(401)

    event_id = request.headers.get("X-ZenRows-Event-Id")
    if event_id in seen_event_ids:
        return "", 200
    seen_event_ids.add(event_id)

    event = request.get_json()
    if event["event_type"] == "run.completed":
        stats = event.get("stats") or {}
        print(f"Run {event['run_id']} of job {event['job_id']} completed: {stats.get('successful')}/{stats.get('total')} succeeded")
        # Download the run's results here, for example with save_snapshot.py.
    elif event["event_type"] == "run.failed":
        print(f"Run {event['run_id']} failed: {event.get('failure_reason')}")
    return "", 200

if __name__ == "__main__":
    app.run(port=8000)

The receiver verifies the raw request body as received, before any JSON parsing, so the signature always matches. The event body carries event_type, job_id, run_id and the run’s stats, so your handler knows which run to download.

In production, keep seen_event_ids in your database instead of memory, and keep the secret in an environment variable or a secret manager.

Pause or delete the schedule

A scheduled job keeps firing, and spending credits, until you pause or delete it. When you’re done with the course, run stop_tracker.py:

Python (SDK)

from zenrows import ZenRowsBatchClient

ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY"

with open("tracker_job_id.txt") as f:
    job_id = f.read().strip()

client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY)
job = client.job(job_id)

paused = job.schedule.pause()
print("Schedule state:", paused.data.schedule_state.value)

job.delete()
print("Deleted job", job_id)

Schedule state: paused
Deleted job 01M4YA87MZSHGRM3SW5093VKNQ

To keep the job and stop it for a while, call job.schedule.pause() alone, and job.schedule.resume() later. To change the cadence, use job.schedule.update(...) with a new builder, see Scheduled jobs.

Ask your AI agent

An agent handles both halves of the tracker: a coding agent sets up the schedule, and any MCP client checks the results whenever you ask. The schedule itself runs in the Zenrows cloud, so nothing needs to stay open on your side. Set up the daily schedule. Paste this prompt into a coding agent such as Claude Code or Cursor, with the Zenrows docs connected. The agent looks up the details and writes the calls.

Using the Zenrows docs, create a scheduled Zenrows Batch job that scrapes every URL in
product_urls.txt every day at 09:00 Madrid time, reading each product's name, price and SKU
with the selectors in PRODUCT_SELECTORS from product_page.py. Read my API key from the
ZENROWS_API_KEY environment variable, then show me the job ID and when the next run starts.

The same prompt schedules a catalog of thousands of products: every URL in the file becomes part of the daily job. Check the latest run. With the Zenrows MCP server connected to Claude, Cursor or any MCP client, paste this prompt with your job ID. Your agent picks the right Zenrows tools and settings on its own.

Use Zenrows to get the latest results of my batch job <job ID> and give me each product's
name and price. Flag any price different from $69.00 for the Abominable Hoodie and
$54.00 for the Hero Hoodie.

What you built

Your tracker now runs itself. Over six lessons you used all five Zenrows primitives in one project:

Lesson	Primitive	What it does in the tracker
1	Fetch	Downloads any page as HTML or markdown, with `mode=auto` choosing the configuration.
2	Extract	Returns a page’s data as JSON with `extract=auto`, next to exact fields from `css_extractor`.
3	Crawl	Follows links from the catalog to collect every product URL.
4	Batch	Scrapes every product URL as one job and writes JSON and CSV.
5	Browser	Drives a cloud browser through a real cart flow.
6	Batch, scheduled	Re-runs the scrape every day and reports each run with a signed webhook.

Where to go next

Turn a whole catalog into a structured dataset

The production version of this course: crawl, scrape and store a full catalog.

Detect changes on any page and get alerted

Compare each day’s snapshot with the last one and alert on what changed.

Choose the right Zenrows primitive

When to reach for Fetch, Extract, Crawl, Batch or Browser on your own project.

Zenrows Academy

Browse every recipe by goal, surface and industry.

Start Here

Foundations Course

Fetch Recipes

Extract Recipes

Crawl Recipes

Batch Recipes

Browser Recipes

Pipelines

Use Cases

By Surface

Production

Lesson 6: Run the tracker every day

Create the daily schedule

Test it with a run now

Save a dated snapshot

Get notified when a run finishes

Pause or delete the schedule

Ask your AI agent

What you built

Where to go next

Turn a whole catalog into a structured dataset

Detect changes on any page and get alerted

Choose the right Zenrows primitive

Zenrows Academy

​Create the daily schedule

​Test it with a run now

​Save a dated snapshot

​Get notified when a run finishes

​Pause or delete the schedule

​Ask your AI agent

​What you built

​Where to go next

Turn a whole catalog into a structured dataset

Detect changes on any page and get alerted

Choose the right Zenrows primitive

Zenrows Academy

Create the daily schedule

Test it with a run now

Save a dated snapshot

Get notified when a run finishes

Pause or delete the schedule

Ask your AI agent

What you built

Where to go next