Access, auth, and limits
Auth is a straightforward API key, accepted either in the query string or as a Bearer header. The query-string form is convenient for clients that cannot set custom headers, though the header form is the one to prefer since URLs leak into logs and shell history more readily than headers do.
The real constraint is that Crawly is a native macOS application, so the crawling happens on your machine and the tool is unavailable on Windows or Linux. The endpoint is getcrawly.com/api/mcp; getcrawly.com/mcp, which appears in some listings, is the marketing page and rejects a POST.
Where it works and what to build
Crawly documents Claude Code, Cursor, Codex and Windsurf, and the API-key auth means any client that can set a header or a query parameter will connect.
The workflow that justifies it is the pre and post deploy diff. Crawl before a release, crawl after, then ask the agent what changed via get_issue_diff, and broken internal links or newly noindexed pages surface immediately instead of a fortnight later in Search Console. Pair that with get_broken_with_inlinks and the agent tells you not just what broke but which pages point at it, which is the part that determines whether it matters.
Which AI clients can use Crawly MCP?
Not every AI assistant supports MCP natively. Per-client picture for Crawly:
| Client | Support | Notes |
|---|---|---|
| Claude Code | Native MCP | — |
| Cursor | Native MCP | — |
| Codex | Native MCP | — |
| Windsurf | Native MCP | — |
How to connect Crawly MCP to Claude Desktop
Add this block to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows), then restart Claude Desktop. Cursor and Codex use a similar schema.
{
"mcpServers": {
"crawly": {
"command": "npx",
"args": ["mcp-remote", "<YOUR_MCP_URL>"]
}
}
}
// HTTP-transport MCP. Get your <YOUR_MCP_URL> from the vendor dashboard.
// Server source: https://www.getcrawly.com/api/mcp
// Claude Code (one-liner): claude mcp add crawly <YOUR_MCP_URL> --transport httpSome MCP servers require an API key. Check the source page for setup: getcrawly.com/api/mcp.
Does Crawly MCP cost extra?
Crawly ships an official MCP server that uses your existing subscription credentials.
- No separate MCP licence to buy. You pay for your Crawly plan, the MCP server queries that account.
- Whether it's gated to a specific tier wasn't extracted automatically — verify the access requirements on the MCP page or pricing docs.
Crawly also publishes a REST API
Crawly publishes an API, but per-tier gating wasn't extracted automatically.
See Crawly API pageWhat you can do with Crawly from an AI agent
Three shapes of agent workflow. Pick the row that matches how you actually use Claude / Cursor / Codex day to day.
Skip the dashboard, ask Claude
- "What keywords moved this week?" answered in seconds
- Quick one-off questions without leaving the editor
- Pipe Crawly data into your personal notes / docs without writing a script
Run client analysis at agent speed
- Hand the MCP server to your team — every consultant can ask Claude about any client's Crawly account
- Multi-account workflows without context-switching the Crawly UI
- Reports you'd normally spend hours on, done before standup
Production agent workflows
- Embed in your internal Codex / Cursor agents that triage Crawly data daily
- Combine with other MCP servers in a multi-step agent chain
- Constrain agents to read-only roles to keep production Crawly safe
Limits and gotchas
- Official MCP server — published by Crawly themselves, so it should track product changes.
- Fallback: the REST API works for any flow the MCP server doesn't cover.
- Rate limits: MCP calls count against your underlying Crawly API quota. Burst usage from a curious agent can drain a daily allowance fast.
Agent-readiness verdict
Remarkable value: a free crawler with a 16-tool MCP surface that covers diffing, migration redirects and internal-link suggestions. The constraint is platform, not capability, since Crawly is macOS-only.
Moderate agent-fit with native MCP support. Crawly includes an official MCP server and documented public API, making it viable for Claude workflows, but lacks OpenAPI specs and SDKs that would streamline integration. The absence of webhooks and structured outputs limits automation potential beyond basic crawl-and-parse tasks.
Scored by Joonas (TMB).
Crawly MCP & API FAQ
Is the Crawly MCP server official?+
Can I use the Crawly MCP with Cursor or Codex?+
Does Crawly have a public API?+
Sources
- Crawly official site: https://www.getcrawly.com/
- MCP server page: https://www.getcrawly.com/api/mcp
- API docs: https://www.getcrawly.com/api
Data verified by Joonas on the dates shown. MCP server status auto-rechecked weekly.
10+ years in digital marketing. I review marketing software for AI-stack fit: real pricing, MCP and API support, and how cleanly each tool drops into an AI agent workflow, cross-checked against verified data and real user feedback.