Tony Wang8 min readBest Web Data Connectors for Claude in 2026
Firecrawl, Bright Data, Zyte and Crawlora as Claude custom connectors: how each handles sign-in, what a probe showed, and what broke when we added ours.
Claude can reach the web in two ways: built-in search, and connectors that give it a vendor's own tools. Firecrawl's October 8, 2026 roundup of fourteen Claude connectors is mostly productivity apps (Asana, Notion, Slack, Stripe) with Firecrawl as the one web-data entry. This post asks the narrower question for people who need data from websites: which web-data vendors can you add as a custom connector, and what actually happens when you do. One of the four is us, and our first attempt did not work.
What we tested, and what we did not
We did three things on October 26, 2026.
- Read the docs. Anthropic's custom connector article, Firecrawl's MCP docs, Zyte's MCP docs and Bright Data's MCP README.
- Probed each hosted endpoint with an unauthenticated
initializerequest and read the OAuth discovery documents it pointed to. These are read-only requests; we registered no clients on anyone's server but our own. - Added Crawlora to claude.ai as a custom connector and ran a search, before and after fixing a bug.
We did not add Firecrawl, Zyte or Bright Data to Claude, so we cannot tell you how their sign-in screens look or how their tools behave inside a chat. We also did not test the static-header route: in the account we tested, Claude did not show a Request headers field, so every Crawlora result below comes from OAuth sign-in.
How Claude adds a custom connector
On Pro and Max, the path is Customize, Connectors, Add, Add custom connector. You enter a name and the server URL, Claude detects the authentication settings, and you can change them before adding. Anthropic's article lists three authentication choices (Sign in now, Sign in when needed, No sign in), three OAuth client choices (Claude's published identity, Register automatically, or your own client) and a Request headers section for fixed credentials on servers that do not use OAuth. Free accounts are limited to one custom connector, and on Team and Enterprise an owner adds the connector once and members connect individually.
Two details decide how smooth a vendor feels:
- A server that answers 401 at the start tells Claude to run sign-in immediately.
- A server that lists its tools without credentials looks connected straight away, and the failure arrives at the first call.
The four vendors
| Vendor | Claude path | Unauthenticated initialize | Registration | Tools | Run by us |
|---|---|---|---|---|---|
| Firecrawl | One-click connector, or the OAuth server https://mcp.firecrawl.dev/v2/mcp-oauth | 401 with OAuth metadata | Dynamic registration, PKCE S256, client ID metadata documents | Search, scrape, crawl, monitor, structured data | Probed only |
| Zyte | Documented custom connector at https://mcp.zyte.com/v1/mcp | 401 with OAuth metadata | Dynamic registration, PKCE S256, client ID metadata documents | HTTP and browser fetch, search, Scrapy Cloud | Probed only |
| Bright Data | Token in the URL (README); OAuth metadata also served | 401 with OAuth metadata | Dynamic registration, PKCE S256 | 32 in the README, filterable by group in the URL | Probed only |
| Crawlora | Custom connector at https://mcp.crawlora.net/mcp/agent | 200; the first tool call returns 401 | Dynamic registration, PKCE S256 | Three on /mcp/agent, thousands on /mcp | Yes, with a bug |
Firecrawl: the one with a one-click connector
Firecrawl's docs show an "Add to Claude" install for its native connector and give the OAuth URL for clients that take a server address. Its docs also describe a keyless tier for search, scrape and parse within daily limits, and an API-key variant at https://mcp.firecrawl.dev/v2/mcp with a bearer token. Our probe got a 401 whose challenge points at OAuth metadata, and its authorization server supports dynamic registration. Choose it when you want the shortest add flow and a broad scraping toolset. We did not run it inside Claude.
Zyte: the only one that documents the claude.ai steps
Zyte's MCP page walks through claude.ai explicitly: Settings, Connectors, Add custom connector, name it Zyte, paste https://mcp.zyte.com/v1/mcp, Connect, sign in. Signing in gives your agent an organization-scoped access key you can use to track and cap spend, and new sign-ins get a $5 trial credit for one month, per its docs. Its tools cover plain HTTP fetches, browser rendering, search and Scrapy Cloud projects. Choose it when you already run Scrapy or Zyte API and want Claude to work inside that account. We did not run it inside Claude.
Bright Data: token in the URL, OAuth quietly available
Bright Data's README says to add https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN and, for Claude Desktop, to paste that URL as a custom connector. A token inside a URL ends up in connector settings and anywhere the URL is logged or copied, so treat it like a password. What the README does not mention is that the same server answers 401 with OAuth metadata and a registration endpoint, so a sign-in flow looks possible; we did not test whether it works with Claude. The README lists 32 tools and lets you narrow them with groups= and tools= URL parameters. Choose it when you need its unblocking infrastructure and are comfortable scoping the toolset with URL parameters.
Crawlora: it works, after we fixed it
Crawlora's connector took two tries, and the first failure is the useful part.
Attempt one, before the fix. Claude showed "Couldn't register with Crawlora's sign-in service" before any consent screen. Our registration endpoint accepts only an allowlist of callback hosts, and it had claude.ai but not claude.com. Anthropic documents both callbacks, and a client that registers both gets the whole registration refused if one is unknown. We confirmed it by sending our endpoint a Claude-shaped request: with both callbacks it returned a 400, and with only claude.ai it succeeded.
The first-run trap. We then added the connector with sign-in switched off. Claude listed three tools and showed it as connected, because our server lets anyone read the tool list. The search ran, search_tools worked, and the real call failed with a 401 and a prompt to reconnect. A connector that looks healthy and then fails is easy to blame on the wrong thing, so we are noting it here.
Attempt two, after the fix. We added the exact claude.com host to the allowlist (lookalike hosts and subdomains stay rejected), deployed it the same day, and tried again. Our server logs for that sign-in show registration, authorization checks, the consent approval and the token exchange all returning success. Claude then ran its two discovery steps and the call, and answered a request for the top three results on "model context protocol" with three links, all returned through Crawlora's web_search.
Connecting the three-tool /mcp/agent endpoint is deliberate. Claude searches tools by keyword, and the default endpoint lists thousands, so the three-tool endpoint keeps discovery to one search call. Claude's permission screen split them into two read-only tools (get a tool, search tools) and one "other" tool (call tool), all set to Needs approval by default.
Choose it when you want many named platforms behind one account and one connector. We did not test the static-key header route, and sign-in uses your Crawlora account, so calls bill to it.
What to check before you rely on a connector
- Does it start sign-in when you add it, or only at the first call? A 401 at the start is clearer than a healthy-looking connector that fails later.
- Where does the credential live? OAuth keeps it out of your settings; a token in a URL or header sits in them.
- How many tools does it list? Keep a chat connector to a handful, or scope it, as Bright Data's URL parameters and Crawlora's /mcp/agent allow.
- Which account pays? Zyte's sign-in creates an organization key; Crawlora's bills your Crawlora account.
- Does it work on your plan? Free accounts get one custom connector.
Which one to pick
- Fastest to add, broad scraping: Firecrawl.
- Already on Scrapy or Zyte API: Zyte, which is the one with written claude.ai steps.
- A named hard site at volume: Bright Data, scoped with its URL parameters.
- Many named platforms behind one sign-in: Crawlora, through
/mcp/agent.
Add Crawlora to Claude
One sign-in, thousands of tools behind a three-tool endpoint. 2,000 free credits a month, no card.
Sources
Related reading
- Best Web Data Plugins for Codex and Cursor in 2026 — the same four vendors as plugins, and why only OAuth servers can ship inside a Codex plugin.
- Best Web Search MCP Servers in 2026 — live-tested search endpoints.
- Can Your AI Coding Agent Use a 3,600-Tool MCP Server? — why a small default tool surface matters.
- Best Web Scraping MCP Servers for Claude Code in 2026 — the terminal-side counterpart.
- Crawlora vs Firecrawl · Crawlora vs Bright Data · Crawlora vs Zyte.
Frequently asked questions
Can I add a web scraping MCP server to Claude as a custom connector?
Yes. Anthropic's custom connector article says you add a remote MCP server by URL, and that authentication can be OAuth sign-in, fixed request headers such as an API key, or no sign-in. On October 26, 2026 we probed Firecrawl, Zyte, Bright Data and Crawlora; all four advertise OAuth with dynamic client registration and PKCE, and we added Crawlora to claude.ai end to end.
Why did Claude say it couldn't register with the sign-in service?
Claude's 'Register automatically' option sends a dynamic client registration, and the server can refuse it. Crawlora's did on October 26, 2026 because its callback allowlist accepted claude.ai but not claude.com, and Anthropic documents both Claude callbacks. Allowing the exact claude.com host fixed it; lookalike hosts and subdomains stay rejected.
Why does my Claude connector show as connected but fail on the first call?
A server that lists its tools without credentials looks connected as soon as you add it, and only rejects the first real tool call. Crawlora does this: with sign-in switched off, Claude listed three tools, then the search call returned a 401 and Claude offered to reconnect. A server that answers 401 at the start makes Claude run sign-in immediately.
How many custom connectors can I add to Claude?
Anthropic's article says free accounts are limited to one custom connector. Pro and Max add connectors from Customize, Connectors; on Team and Enterprise an owner adds the connector in organization settings and members then connect individually.
Which connector did you actually test inside Claude?
Only Crawlora, on October 26, 2026. For Firecrawl, Zyte and Bright Data we read their documentation and sent unauthenticated requests to their hosted endpoints, but did not add them to Claude. We also did not test the static request-header route, because the Request headers field did not appear in the account we used.