> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Claude Agent SDK
> Build autonomous, multi-step AI agents that automate and scrape any site via ZenRows' MCP and the Claude Agent SDK. No orchestration code.
Combine the Claude Agent SDK with the [ZenRows hosted MCP server](/integrations/mcp/mcp-overview) to build autonomous, multi-step agents that can read any website in real time. The Agent SDK has native support for MCP servers through its `mcp_servers` option, so plugging in ZenRows requires a single block of code, with no tool-use boilerplate, no scraping infrastructure, and no anti-bot tuning.
This guide covers the Claude Agent SDK. If you want to use ZenRows with the standard Anthropic API (Messages API, tool use, structured outputs, MCP connector), use the [ZenRows MCP server](/integrations/mcp/mcp-overview) with Anthropic's MCP connector.
## What is Claude Agent SDK?
The Claude Agent SDK is Anthropic's Python and TypeScript library for building production AI agents using the same agent loop, tools, and context management that power Claude Code. It exposes two main entry points: `query()` for stateless one-shot agent runs, and `ClaudeSDKClient` for stateful, interactive sessions. Both support MCP servers, custom Python tools, subagents, hooks, and Claude Code's built-in tools (`Read`, `Write`, `Edit`, `Bash`, `WebSearch`, `WebFetch`, and more).
Agents built with the SDK are well-suited to tasks that need multiple steps, autonomous tool selection, or long-running workflows. Adding ZenRows MCP gives every agent in your workflow real-time access to any website, including JavaScript-heavy and anti-bot-protected ones.
### Key benefits
* **Native MCP support, zero glue code**
The Agent SDK's `mcp_servers` option takes the ZenRows server URL and your API key. The agent then sees ZenRows' tools (`scrape` and 30+ browser automation tools) automatically, with no tool use loop to maintain.
* **Autonomous, multi-step web tasks**
Agents can scrape one page, decide what to scrape next based on what they read, and chain together browser interactions like form fills and clicks across multiple pages, without you writing the orchestration loop.
* **Anti-bot bypass on every tool call**
Premium residential proxies, JavaScript rendering, and stealth fingerprinting are built-in standards. Your agents reach JavaScript-heavy SPAs, Cloudflare-protected sites, and geo-restricted pages without configuration.
* **Composes with the full Agent SDK feature set**
ZenRows tools work alongside Claude Code's built-in tools, custom Python tools, subagents, hooks, and permission gates. Pass them to a single agent or many; the integration is the same.
* **Hosted execution, no local installation**
The hosted MCP server runs on ZenRows infrastructure. Your agent code stays clean, with no proxy management, scraping retries, or headless browser orchestration in your codebase.
### Use cases
The Agent SDK and ZenRows combination unlocks a wide range of agent workflows:
Agents that visit multiple URLs, follow citations, and synthesize findings from across the web into a single brief.
Agents that take a company name, scrape the company website, find pricing pages and product launches, and write enriched profiles into your CRM.
Agents that periodically check competitor pricing pages, blog posts, and changelogs, then surface what changed since the last run.
A research subagent that gathers information and hands off to a writer subagent that produces the final report, both pulling from live web data.
Agents that navigate multi-step checkout flows, login-walled forms, or job applications using ZenRows' browser tools.
## Getting started: Basic usage
A minimal working example: a single agent equipped with ZenRows MCP that can answer questions by reading any website in real time.
Python 3.10 or later is required.
```shell theme={null}
pip install claude-agent-sdk
```
```shell theme={null}
ZENROWS_API_KEY=your_zenrows_key
ANTHROPIC_API_KEY=your_anthropic_key
```
Find your ZenRows API key in the [ZenRows dashboard](https://app.zenrows.com/register). Find your Anthropic API key in the Claude Console under **Settings > API keys**.
```python Python expandable theme={null}
import asyncio
import os
from claude_agent_sdk import query, ClaudeAgentOptions
from dotenv import load_dotenv
load_dotenv()
ZENROWS_API_KEY = os.environ["ZENROWS_API_KEY"]
async def main() -> None:
options = ClaudeAgentOptions(
model="claude-sonnet-4-6",
system_prompt=(
"You are a research assistant with live web access. "
"Use the ZenRows tools to scrape pages when you need real-time information."
),
mcp_servers={
"zenrows": {
"type": "http",
"url": "https://mcp.zenrows.com/mcp",
"headers": {
"Authorization": f"Bearer {ZENROWS_API_KEY}",
},
}
},
allowed_tools=["mcp__zenrows__*"],
permission_mode="bypassPermissions",
)
async for message in query(
prompt="Visit https://huggingface.co/blog and summarize the three most recent posts.",
options=options,
):
if hasattr(message, "result"):
print(message.result)
asyncio.run(main())
```
Here's what each part does:
* `ClaudeAgentOptions` configures the agent. The `mcp_servers` dictionary registers the ZenRows hosted MCP server with `type: "http"`, the server URL, and an `Authorization: Bearer` header carrying your ZenRows API key as the OAuth token.
* `allowed_tools=["mcp__zenrows__*"]` pre-approves every tool from the ZenRows server. The Agent SDK names MCP tools using the pattern `mcp____`, so the wildcard enables `mcp__zenrows__scrape`, `mcp__zenrows__browser_navigate`, and so on.
* `permission_mode="bypassPermissions"` lets the agent call tools without an interactive approval prompt. Use this for unattended scripts and production agents. For interactive use, leave it at the default and the SDK will prompt before each tool call.
* `query()` runs the agent loop. The agent discovers ZenRows' tools, picks the right one for the prompt, calls it, and synthesizes the answer. Messages stream back asynchronously; the final answer arrives as a message with a `result` attribute.
## Advanced usage: Building an autonomous research agent
The real power of the Agent SDK is letting an agent decide for itself which tools to call across multiple steps. With ZenRows MCP, that means the agent can autonomously read several pages, follow relevant links, and synthesize findings, all from a single prompt.
The example below builds a research agent that visits a blog page, identifies the most relevant linked articles, scrapes each one, and produces a synthesized report:
```python Python expandable theme={null}
import asyncio
import os
from claude_agent_sdk import query, ClaudeAgentOptions
from dotenv import load_dotenv
load_dotenv()
ZENROWS_API_KEY = os.environ["ZENROWS_API_KEY"]
async def main() -> None:
options = ClaudeAgentOptions(
model="claude-opus-4-7",
system_prompt=(
"You are a thorough web researcher. When given a topic and a starting URL, "
"visit the URL, identify the 3 most relevant linked articles, scrape each one, "
"and produce a synthesized report covering key findings, common themes, and "
"any contradictions. Cite each source with its URL."
),
mcp_servers={
"zenrows": {
"type": "http",
"url": "https://mcp.zenrows.com/mcp",
"headers": {
"Authorization": f"Bearer {ZENROWS_API_KEY}",
},
}
},
allowed_tools=["mcp__zenrows__*"],
permission_mode="bypassPermissions",
max_turns=20,
)
async for message in query(
prompt=(
"Research recent developments in open-source LLMs. "
"Start at https://huggingface.co/blog and follow the most relevant links."
),
options=options,
):
if hasattr(message, "result"):
print(message.result)
asyncio.run(main())
```
* Don't write an orchestration loop in your code. The agent decides on its own when to scrape, which URLs to follow, and when it has enough information to write the report. The SDK runs the multi-step loop automatically.
* Use a capable model for multi-step tasks. `claude-opus-4-7` is the right default for autonomous workflows because it plans its next action based on what it has learned so far. For lighter tasks, `claude-sonnet-4-6` is faster and cheaper.
* `max_turns=20` caps the total number of agent turns. This is a useful safety net that prevents indefinite loops. Adjust upward for deeper research, downward for tighter workflows.
* Same MCP config as the basic example. The only things that changed are the model, the system prompt, and the prompt itself.
## Stateful sessions with ClaudeSDKClient
For agents that need to maintain conversation state across multiple user inputs (chatbots, interactive research assistants, follow-up questions), use `ClaudeSDKClient` instead of `query()`. The client supports bidirectional, interactive conversations with the same MCP configuration.
```python Python expandable theme={null}
import asyncio
import os
from claude_agent_sdk import ClaudeSDKClient, ClaudeAgentOptions
from dotenv import load_dotenv
load_dotenv()
ZENROWS_API_KEY = os.environ["ZENROWS_API_KEY"]
async def main() -> None:
options = ClaudeAgentOptions(
model="claude-sonnet-4-6",
system_prompt="You are a web research assistant with access to ZenRows scraping tools.",
mcp_servers={
"zenrows": {
"type": "http",
"url": "https://mcp.zenrows.com/mcp",
"headers": {
"Authorization": f"Bearer {ZENROWS_API_KEY}",
},
}
},
allowed_tools=["mcp__zenrows__*"],
permission_mode="bypassPermissions",
)
async with ClaudeSDKClient(options=options) as client:
# First turn
await client.query("Visit https://huggingface.co/blog and summarize the top post.")
async for message in client.receive_response():
if hasattr(message, "result"):
print(message.result)
# Follow-up turn, sharing conversation history
await client.query("Now scrape the linked GitHub repo and tell me what license it uses.")
async for message in client.receive_response():
if hasattr(message, "result"):
print(message.result)
asyncio.run(main())
```
The client preserves conversation context between turns, so the agent remembers what it scraped in the first turn when answering the follow-up. Use `query()` for one-shot scripts and `ClaudeSDKClient` whenever you need multi-turn conversation state.
## Combining with custom Python tools
The Agent SDK also supports custom Python tools defined as in-process MCP servers. These can be used alongside the ZenRows hosted MCP server in the same agent.
```python Python expandable theme={null}
import asyncio
import os
from claude_agent_sdk import (
query,
tool,
create_sdk_mcp_server,
ClaudeAgentOptions,
)
from dotenv import load_dotenv
load_dotenv()
ZENROWS_API_KEY = os.environ["ZENROWS_API_KEY"]
# Custom in-process tool
@tool("save_summary", "Save a research summary to a file", {"filename": str, "content": str})
async def save_summary(args):
with open(args["filename"], "w") as f:
f.write(args["content"])
return {
"content": [
{"type": "text", "text": f"Saved to {args['filename']}"}
]
}
local_server = create_sdk_mcp_server(
name="local",
version="1.0.0",
tools=[save_summary],
)
async def main() -> None:
options = ClaudeAgentOptions(
model="claude-sonnet-4-6",
system_prompt=(
"Research the user's topic with ZenRows, then save your final summary "
"using the save_summary tool."
),
mcp_servers={
"zenrows": {
"type": "http",
"url": "https://mcp.zenrows.com/mcp",
"headers": {
"Authorization": f"Bearer {ZENROWS_API_KEY}",
},
},
"local": local_server,
},
allowed_tools=[
"mcp__zenrows__*",
"mcp__local__save_summary",
],
permission_mode="bypassPermissions",
)
async for message in query(
prompt=(
"Scrape https://huggingface.co/blog, summarize the top three posts, "
"and save the summary to hf_summary.txt."
),
options=options,
):
if hasattr(message, "result"):
print(message.result)
asyncio.run(main())
```
This pattern composes two MCP servers in the same agent: ZenRows for live web access and an in-process Python server for local file writes. The agent autonomously chains them, scraping the blog first and then calling `save_summary` with the finished text.
## Troubleshooting
### 424 failed dependency or 401 unauthorized from the MCP server
1. Confirm the `Authorization: Bearer YOUR_ZENROWS_API_KEY` header is present in the `headers` dict on the MCP server entry.
2. Verify your API key in the [ZenRows dashboard](https://app.zenrows.com/account/settings) and confirm your subscription has remaining quota.
3. Confirm the server URL is exactly `https://mcp.zenrows.com/mcp` with `type: "http"`.
### Agent connects but never calls ZenRows tools
1. Confirm `allowed_tools` includes the ZenRows tool names. Without an entry matching `mcp__zenrows__*` (or specific tool names), the SDK does not expose the tools to the agent.
2. If `permission_mode` is at the default, the SDK prompts before each tool call. For unattended scripts, set `permission_mode="bypassPermissions"`.
3. Strengthen the system prompt to explicitly point the agent at the ZenRows tools, for example, "Use the ZenRows tools to fetch any URL the user mentions."
### Agent repeatedly scrapes the same page
Add explicit guidance in the system prompt about what to do once it has the data, for example, "After scraping each URL once, summarize and stop." For complex flows, lower `max_turns` to bound runaway agents.
### Page blocked or data missing in scraped content
If the agent connects and invokes tools correctly but the scraped content comes back blocked or incomplete, the issue is from the target page rather than the MCP connection. Add explicit guidance to your prompt, for example "Scrape this page using JavaScript rendering" or "Use Premium Proxies." See the [Universal Scraper API troubleshooting guide](/universal-scraper-api/troubleshooting/troubleshooting-guide) for parameter-level fixes.
### `ANTHROPIC_API_KEY` not found
The Agent SDK reads `ANTHROPIC_API_KEY` from the environment. Confirm the variable is set in your shell or `.env` file. For Bedrock, Vertex AI, or Azure routing, see the Claude Agent SDK authentication docs.
## FAQ (Frequently Asked Questions)
The standard Anthropic Claude integration uses the Messages API, tool use, structured outputs, and the MCP connector. In this pattern, you make one or two model calls and process the results, with the tool-use loop running in your code.
The Claude Agent SDK is for autonomous, multi-step agents. The SDK runs the orchestration loop, decides which tools to call, and chains tool calls without you having to write the loop. It also ships with Claude Code's built-in tools (file editing, shell, web search/fetch) and supports subagents and hooks. Use the standard integration when you need direct control. Use the Agent SDK when you want autonomy, multi-step behavior, and Claude Code's broader toolset.
All current Claude 4 models. For autonomous multi-step workflows, `claude-opus-4-7` is the strongest. For balanced cost-performance, `claude-sonnet-4-6` is a good default. For high-throughput, cost-sensitive workloads, `claude-haiku-4-5` is a faster, cheaper option.
Yes. Use the `command` and `args` form instead of `type: "http"`:
```python Python theme={null}
mcp_servers={
"zenrows": {
"command": "npx",
"args": ["-y", "@zenrows/mcp"],
"env": {"ZENROWS_API_KEY": ZENROWS_API_KEY},
}
}
```
The hosted HTTP URL is recommended because it requires no local installation and runs the same way in development and production. See the [ZenRows MCP overview](/integrations/mcp/mcp-overview) for the local stdio configuration.
Yes. Build the `mcp_servers` dict and `allowed_tools` list once and reuse them across as many `ClaudeAgentOptions` instances as you need. Each agent gets independent tool calls, and ZenRows handles them all on the same hosted server.
Yes. Both `query()` and `ClaudeSDKClient.receive_response()` are async iterators that stream messages as the agent works. Each message can be a tool call, a tool result, a partial text response, or the final result. Use `if hasattr(message, "result")` to filter for the final answer, or inspect all messages for full traces.
Yes. Add the built-in tool names to `allowed_tools` alongside the ZenRows wildcards. This is the right pattern for agents that need to scrape the web, write to local files, run shell commands, or edit code. See the Combining with custom Python tools section above for a similar pattern with in-process MCP servers.
The Claude Agent SDK and OpenAI Agents SDK solve similar problems with different design philosophies. The Claude Agent SDK ships as "Claude Code as a library," so it includes a rich set of built-in tools (file editing, shell, web search/fetch) and is tightly coupled with the Claude model family. The OpenAI Agents SDK is a thinner orchestration layer optimized for the OpenAI Responses API and GPT models, with a strong handoff and tracing story. ZenRows MCP works with both — pick the SDK that fits your model and toolset preferences. See the [OpenAI Agents SDK integration guide](/integrations/mcp/openai-agents-sdk) for the OpenAI equivalent of this doc.