Combine the Claude Agent SDK with the ZenRows hosted MCP server to build autonomous, multi-step agents that can read any website in real time. The Agent SDK has native support for MCP servers through itsDocumentation Index
Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
Use this file to discover all available pages before exploring further.
mcp_servers option, so plugging in ZenRows requires a single block of code, with no tool-use boilerplate, no scraping infrastructure, and no anti-bot tuning.
This guide covers the Claude Agent SDK. If you want to use ZenRows with the standard Anthropic API (Messages API, tool use, structured outputs, MCP connector), use the ZenRows MCP server with Anthropic’s MCP connector.
What is Claude Agent SDK?
The Claude Agent SDK is Anthropic’s Python and TypeScript library for building production AI agents using the same agent loop, tools, and context management that power Claude Code. It exposes two main entry points:query() for stateless one-shot agent runs, and ClaudeSDKClient for stateful, interactive sessions. Both support MCP servers, custom Python tools, subagents, hooks, and Claude Code’s built-in tools (Read, Write, Edit, Bash, WebSearch, WebFetch, and more).
Agents built with the SDK are well-suited to tasks that need multiple steps, autonomous tool selection, or long-running workflows. Adding ZenRows MCP gives every agent in your workflow real-time access to any website, including JavaScript-heavy and anti-bot-protected ones.
Key benefits
-
Native MCP support, zero glue code
The Agent SDK’smcp_serversoption takes the ZenRows server URL and your API key. The agent then sees ZenRows’ tools (scrapeand 30+ browser automation tools) automatically, with no tool use loop to maintain. -
Autonomous, multi-step web tasks
Agents can scrape one page, decide what to scrape next based on what they read, and chain together browser interactions like form fills and clicks across multiple pages, without you writing the orchestration loop. -
Anti-bot bypass on every tool call
Premium residential proxies, JavaScript rendering, and stealth fingerprinting are built-in standards. Your agents reach JavaScript-heavy SPAs, Cloudflare-protected sites, and geo-restricted pages without configuration. -
Composes with the full Agent SDK feature set
ZenRows tools work alongside Claude Code’s built-in tools, custom Python tools, subagents, hooks, and permission gates. Pass them to a single agent or many; the integration is the same. -
Hosted execution, no local installation
The hosted MCP server runs on ZenRows infrastructure. Your agent code stays clean, with no proxy management, scraping retries, or headless browser orchestration in your codebase.
Use cases
The Agent SDK and ZenRows combination unlocks a wide range of agent workflows:Autonomous research agents
Autonomous research agents
Lead enrichment pipelines
Lead enrichment pipelines
Competitive monitoring
Competitive monitoring
Multi-agent web automation
Multi-agent web automation
Form-filling and extraction agents
Form-filling and extraction agents
Getting started: Basic usage
A minimal working example: a single agent equipped with ZenRows MCP that can answer questions by reading any website in real time.Create a .env file with your API keys
Run the following script
ClaudeAgentOptionsconfigures the agent. Themcp_serversdictionary registers the ZenRows hosted MCP server withtype: "http", the server URL, and anAuthorization: Bearerheader carrying your ZenRows API key as the OAuth token.allowed_tools=["mcp__zenrows__*"]pre-approves every tool from the ZenRows server. The Agent SDK names MCP tools using the patternmcp__<server-name>__<tool-name>, so the wildcard enablesmcp__zenrows__scrape,mcp__zenrows__browser_navigate, and so on.permission_mode="bypassPermissions"lets the agent call tools without an interactive approval prompt. Use this for unattended scripts and production agents. For interactive use, leave it at the default and the SDK will prompt before each tool call.query()runs the agent loop. The agent discovers ZenRows’ tools, picks the right one for the prompt, calls it, and synthesizes the answer. Messages stream back asynchronously; the final answer arrives as a message with aresultattribute.
Advanced usage: Building an autonomous research agent
The real power of the Agent SDK is letting an agent decide for itself which tools to call across multiple steps. With ZenRows MCP, that means the agent can autonomously read several pages, follow relevant links, and synthesize findings, all from a single prompt. The example below builds a research agent that visits a blog page, identifies the most relevant linked articles, scrapes each one, and produces a synthesized report:Stateful sessions with ClaudeSDKClient
For agents that need to maintain conversation state across multiple user inputs (chatbots, interactive research assistants, follow-up questions), useClaudeSDKClient instead of query(). The client supports bidirectional, interactive conversations with the same MCP configuration.
query() for one-shot scripts and ClaudeSDKClient whenever you need multi-turn conversation state.
Combining with custom Python tools
The Agent SDK also supports custom Python tools defined as in-process MCP servers. These can be used alongside the ZenRows hosted MCP server in the same agent.save_summary with the finished text.
Troubleshooting
424 failed dependency or 401 unauthorized from the MCP server
- Confirm the
Authorization: Bearer YOUR_ZENROWS_API_KEYheader is present in theheadersdict on the MCP server entry. - Verify your API key in the ZenRows dashboard and confirm your subscription has remaining quota.
- Confirm the server URL is exactly
https://mcp.zenrows.com/mcpwithtype: "http".
Agent connects but never calls ZenRows tools
- Confirm
allowed_toolsincludes the ZenRows tool names. Without an entry matchingmcp__zenrows__*(or specific tool names), the SDK does not expose the tools to the agent. - If
permission_modeis at the default, the SDK prompts before each tool call. For unattended scripts, setpermission_mode="bypassPermissions". - Strengthen the system prompt to explicitly point the agent at the ZenRows tools, for example, “Use the ZenRows tools to fetch any URL the user mentions.”
Agent repeatedly scrapes the same page
Add explicit guidance in the system prompt about what to do once it has the data, for example, “After scraping each URL once, summarize and stop.” For complex flows, lowermax_turns to bound runaway agents.
Page blocked or data missing in scraped content
If the agent connects and invokes tools correctly but the scraped content comes back blocked or incomplete, the issue is from the target page rather than the MCP connection. Add explicit guidance to your prompt, for example “Scrape this page using JavaScript rendering” or “Use Premium Proxies.” See the Universal Scraper API troubleshooting guide for parameter-level fixes.ANTHROPIC_API_KEY not found
The Agent SDK reads ANTHROPIC_API_KEY from the environment. Confirm the variable is set in your shell or .env file. For Bedrock, Vertex AI, or Azure routing, see the Claude Agent SDK authentication docs.
FAQ (Frequently Asked Questions)
What is the difference between this and the standard Anthropic Claude integration?
What is the difference between this and the standard Anthropic Claude integration?
Which Claude models work with the Agent SDK?
Which Claude models work with the Agent SDK?
claude-opus-4-7 is the strongest. For balanced cost-performance, claude-sonnet-4-6 is a good default. For high-throughput, cost-sensitive workloads, claude-haiku-4-5 is a faster, cheaper option.Can I use the local stdio version of ZenRows MCP with the Agent SDK?
Can I use the local stdio version of ZenRows MCP with the Agent SDK?
command and args form instead of type: "http":Can multiple agents share the same ZenRows MCP configuration?
Can multiple agents share the same ZenRows MCP configuration?
Is streaming supported?
Is streaming supported?
query() and ClaudeSDKClient.receive_response() are async iterators that stream messages as the agent works. Each message can be a tool call, a tool result, a partial text response, or the final result. Use if hasattr(message, "result") to filter for the final answer, or inspect all messages for full traces.Can I combine ZenRows MCP with Claude Code's built-in tools (Read, Write, Bash, etc.)?
Can I combine ZenRows MCP with Claude Code's built-in tools (Read, Write, Bash, etc.)?
allowed_tools alongside the ZenRows wildcards. This is the right pattern for agents that need to scrape the web, write to local files, run shell commands, or edit code. See the Combining with custom Python tools section above for a similar pattern with in-process MCP servers.How does this compare to the OpenAI Agents SDK?
How does this compare to the OpenAI Agents SDK?