> ## Documentation Index > Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > The Zenrows API (Fetch, Extract via extract=auto, and Batch) is described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Use the spec for exact parameter names and types. # Lesson 6: Run the tracker every day > Foundations lesson 6: run the product tracker every day with a scheduled Zenrows Batch job, save each day's data and get notified when it finishes. In [Lesson 4](/academy/foundations/scrape-at-scale) you scraped every product once. A tracker earns its name when it runs on its own. In this last lesson you hand the job to Zenrows as a daily schedule, collect each day's results as a dated snapshot, and learn how to get notified when a run finishes. **What you'll build:** * `schedule_tracker.py` creates a scheduled Batch job that scrapes every product at 09:00 each day. * `run_now.py` fires the job immediately so you can test it. * `save_snapshot.py` writes the latest run to `products-YYYY-MM-DD.json`. * `stop_tracker.py` pauses and deletes the schedule. * Optionally, `webhook_receiver.py` gets a signed `run.completed` event after every run. **What you'll learn:** * How a scheduled job stores its URLs as a template and creates a new run on every fire. * How to set a calendar schedule with a timezone. * How to verify a signed webhook from Zenrows. **Time:** about 10 minutes. ## Create the daily schedule A scheduled Batch job keeps its tasks and `zenrows_params` as a template. Each time the schedule fires, Zenrows starts a fresh run of those tasks, the same kind of run you waited on in Lesson 4: one Fetch request per URL. You set the cadence with one of three builders from `zenrows.batch`: `Rate` (every 60 minutes or more, in minute, hour or day units), `Calendar` (times of day on a daily, weekly or monthly cadence, in a timezone) or `At` (one future time). Create `schedule_tracker.py`. It reuses `product_urls.txt` from Lesson 3 and `PRODUCT_SELECTORS` from Lesson 2: ```python Python (SDK) theme={"dark"} import json from zenrows import ZenRowsBatchClient from zenrows.batch import Calendar, Daily ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY" PRODUCT_SELECTORS = { "name": "h1.product_title", "price": "p.price", "sku": "span.sku", "category": "span.posted_in a", "sizes": "select#size option:not(:first-child) @value", "colors": "select#color option:not(:first-child) @value", } with open("product_urls.txt") as f: product_urls = [line.strip() for line in f if line.strip()] client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY) job = client.submit_scheduled( Calendar(times_of_day=["09:00"], cadence=Daily(), timezone="Europe/Madrid"), [{"url": url, "external_id": url.rstrip("/").split("/")[-1]} for url in product_urls], zenrows_params={"mode": "auto", "css_extractor": json.dumps(PRODUCT_SELECTORS)}, name="product-catalog-tracker-daily", ) with open("tracker_job_id.txt", "w") as f: f.write(job.job_id) details = job.load().data print("Scheduled job:", job.job_id) print("Schedule state:", details.schedule_state.value) print("Next run at:", details.next_scheduled_run) ``` Run `python schedule_tracker.py`: ```text theme={"dark"} Scheduled job: 01M4YA87MZSHGRM3SW5093VKNQ Schedule state: active Next run at: 2026-10-15 07:00:00+00:00 ``` The next run is 09:00 Madrid time, shown in UTC. The script saves the job ID to `tracker_job_id.txt` so the next scripts can find the job. Set `timezone` to any IANA timezone name, and `times_of_day` to full hours in 24-hour format, such as `"09:00"` or `"18:00"`. ## Test it with a run now You don't have to wait until tomorrow. `rerun()` on a scheduled job that hasn't fired yet starts a run from its template right away. Create `run_now.py`: ```python Python (SDK) theme={"dark"} from zenrows import ZenRowsBatchClient ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY" with open("tracker_job_id.txt") as f: job_id = f.read().strip() client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY) run = client.job(job_id).rerun() print("Fired run", run.run_id) run = run.wait(timeout=900) print(f"Run {run.data.status.value}: {run.stats.successful}/{run.stats.total} succeeded") ``` `client.job(job_id)` points at the job by ID without fetching it first. Run `python run_now.py`: ```text theme={"dark"} Fired run 01M4YA893ZADACHR1KKMDS11VR Run completed: 42/42 succeeded ``` ## Save a dated snapshot Each run's results stay available after it finishes. `save_snapshot.py` reads the job's latest run and writes it to a file named after today's date, so every day adds one snapshot you can compare with the day before: ```python Python (SDK) theme={"dark"} import json from datetime import date from zenrows import ZenRowsBatchClient ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY" def as_list(value): if not value: return [] return value if isinstance(value, list) else [value] with open("tracker_job_id.txt") as f: job_id = f.read().strip() client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY) run = client.job(job_id).run.load() print("Latest run:", run.run_id, run.data.status.value) products = [] for result in run.download_to_memory(status="successful"): fields = json.loads(result.body) products.append({ "slug": result.external_id, "name": fields.get("name"), "price": fields.get("price"), "sku": fields.get("sku"), "category": fields.get("category"), "sizes": ", ".join(as_list(fields.get("sizes"))), "colors": ", ".join(as_list(fields.get("colors"))), }) products.sort(key=lambda p: p["slug"]) filename = f"products-{date.today().isoformat()}.json" with open(filename, "w") as f: json.dump(products, f, indent=2) print(f"Saved {len(products)} products to {filename}") ``` Run `python save_snapshot.py`: ```text theme={"dark"} Latest run: 01M4YA893ZADACHR1KKMDS11VR completed Saved 42 products to products-2026-10-15.json ``` Run `save_snapshot.py` after each daily run, by hand, from a cron job, or from the webhook in the next section. ## Get notified when a run finishes A webhook tells your app the moment a run ends, so it can save the snapshot without polling. Zenrows POSTs a `run.completed` event when a run completes and a `run.failed` event when it fails. With `signature: True`, each delivery is signed with your organization's HMAC key, so your receiver can prove it came from Zenrows. Your receiver needs a public HTTPS URL. This script creates your organization's HMAC key, prints its secret (Zenrows returns it once, so store it now) and attaches the webhook to the tracker job: ```python Python (SDK) theme={"dark"} from zenrows import ZenRowsBatchClient ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY" with open("tracker_job_id.txt") as f: job_id = f.read().strip() client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY) key = client.rotate_hmac_key() print("Key id:", key.kid) print("Secret:", key.secret) client.put_job_webhook(job_id, {"url": "https://your-app.com/webhooks/zenrows", "signature": True}) print("Webhook attached to", job_id) ``` If your organization already has a key, reuse its secret and skip `rotate_hmac_key()`. See [Manage HMAC keys](/batch/developer-guide-restapi#manage-hmac-keys). Each signed delivery carries an `X-Signature` header in the form `t=,v1=,kid=`. Recompute `HMAC-SHA256(secret, ".")` with the base64-decoded secret and compare it to `v1`. Deliveries arrive at least once, so skip any `X-ZenRows-Event-Id` you've already handled. This Flask receiver (`pip install flask`) does both: ```python Python (Flask) theme={"dark"} import base64 import hashlib import hmac from flask import Flask, abort, request # Key id -> the base64 secret Zenrows returned when you created the HMAC key. SECRETS = {"YOUR_HMAC_KEY_ID": "YOUR_HMAC_SECRET"} seen_event_ids = set() app = Flask(__name__) def is_signed_by_zenrows(raw_body, signature_header): parts = dict(item.split("=", 1) for item in signature_header.split(",") if "=" in item) secret_b64 = SECRETS.get(parts.get("kid")) if not secret_b64 or "t" not in parts: return False secret = base64.b64decode(secret_b64) expected = hmac.new(secret, f"{parts['t']}.".encode() + raw_body, hashlib.sha256).hexdigest() return hmac.compare_digest(expected, parts.get("v1", "")) @app.post("/webhooks/zenrows") def zenrows_webhook(): if not is_signed_by_zenrows(request.get_data(), request.headers.get("X-Signature", "")): abort(401) event_id = request.headers.get("X-ZenRows-Event-Id") if event_id in seen_event_ids: return "", 200 seen_event_ids.add(event_id) event = request.get_json() if event["event_type"] == "run.completed": stats = event.get("stats") or {} print(f"Run {event['run_id']} of job {event['job_id']} completed: {stats.get('successful')}/{stats.get('total')} succeeded") # Download the run's results here, for example with save_snapshot.py. elif event["event_type"] == "run.failed": print(f"Run {event['run_id']} failed: {event.get('failure_reason')}") return "", 200 if __name__ == "__main__": app.run(port=8000) ``` The receiver verifies the raw request body as received, before any JSON parsing, so the signature always matches. The event body carries `event_type`, `job_id`, `run_id` and the run's `stats`, so your handler knows which run to download. In production, keep `seen_event_ids` in your database instead of memory, and keep the secret in an environment variable or a secret manager. ## Pause or delete the schedule A scheduled job keeps firing, and spending credits, until you pause or delete it. When you're done with the course, run `stop_tracker.py`: ```python Python (SDK) theme={"dark"} from zenrows import ZenRowsBatchClient ZENROWS_API_KEY = "YOUR_ZENROWS_API_KEY" with open("tracker_job_id.txt") as f: job_id = f.read().strip() client = ZenRowsBatchClient(api_key=ZENROWS_API_KEY) job = client.job(job_id) paused = job.schedule.pause() print("Schedule state:", paused.data.schedule_state.value) job.delete() print("Deleted job", job_id) ``` ```text theme={"dark"} Schedule state: paused Deleted job 01M4YA87MZSHGRM3SW5093VKNQ ``` To keep the job and stop it for a while, call `job.schedule.pause()` alone, and `job.schedule.resume()` later. To change the cadence, use `job.schedule.update(...)` with a new builder, see [Scheduled jobs](/batch/sdk/python#scheduled-jobs). ## Ask your AI agent An agent handles both halves of the tracker: a coding agent sets up the schedule, and any MCP client checks the results whenever you ask. The schedule itself runs in the Zenrows cloud, so nothing needs to stay open on your side. **Set up the daily schedule.** Paste this prompt into a coding agent such as Claude Code or Cursor, with the [Zenrows docs connected](/academy/connect-your-ai-agent#give-your-agent-the-zenrows-docs). The agent looks up the details and writes the calls. ```text theme={"dark"} Using the Zenrows docs, create a scheduled Zenrows Batch job that scrapes every URL in product_urls.txt every day at 09:00 Madrid time, reading each product's name, price and SKU with the selectors in PRODUCT_SELECTORS from product_page.py. Read my API key from the ZENROWS_API_KEY environment variable, then show me the job ID and when the next run starts. ``` The same prompt schedules a catalog of thousands of products: every URL in the file becomes part of the daily job. **Check the latest run.** With the [Zenrows MCP server](/mcp/overview) connected to Claude, Cursor or any MCP client, paste this prompt with your job ID. Your agent picks the right Zenrows tools and settings on its own. ```text theme={"dark"} Use Zenrows to get the latest results of my batch job and give me each product's name and price. Flag any price different from $69.00 for the Abominable Hoodie and $54.00 for the Hero Hoodie. ``` ## What you built Your tracker now runs itself. Over six lessons you used all five Zenrows primitives in one project: | Lesson | Primitive | What it does in the tracker | | - | - | - | | [1](/academy/foundations/first-request) | **Fetch** | Downloads any page as HTML or markdown, with `mode=auto` choosing the configuration. | | [2](/academy/foundations/extract-product-data) | **Extract** | Returns a page's data as JSON with `extract=auto`, next to exact fields from `css_extractor`. | | [3](/academy/foundations/discover-every-product) | **Crawl** | Follows links from the catalog to collect every product URL. | | [4](/academy/foundations/scrape-at-scale) | **Batch** | Scrapes every product URL as one job and writes JSON and CSV. | | [5](/academy/foundations/automate-a-browser) | **Browser** | Drives a cloud browser through a real cart flow. | | 6 | **Batch, scheduled** | Re-runs the scrape every day and reports each run with a signed webhook. | ## Where to go next The production version of this course: crawl, scrape and store a full catalog. Compare each day's snapshot with the last one and alert on what changed. When to reach for Fetch, Extract, Crawl, Batch or Browser on your own project. Browse every recipe by goal, surface and industry. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.