Data · Local MCP server

Crawl Readiness

Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.

Visit Crawl Readiness's website

What the MCP Registry states

The entry as published to the official MCP Registry (read 2026-10-04), latest version.

Registry name
io.github.Grubbomatic/crawl-readiness
Version
0.2.4
Status
Active
Category
data
Transport
stdio (local process)
Package
npm
Published
2026-09-30
Updated
2026-09-30
Publisher
Grubbomatic (GitHub)
Website
www.crawlreadiness.com/mcp
Repository
github.com/Grubbomatic/crawl-readiness-mcp
Source
Registry API entry

Packages

RegistryPackageVersionTransportEnvironment variables
npmcrawl-readiness-mcp0.2.4stdioCRAWL_READINESS_API_KEY (secret), CRAWL_READINESS_API_BASE

How to connect Crawl Readiness

Crawl Readiness runs locally from a Node.js package published to the npm registry: crawl-readiness-mcp version 0.2.4. It speaks MCP over stdio, so the client starts it as a program and talks to it through standard input and output. It needs Node.js; clients usually start it with npx — the usual command is npx -y crawl-readiness-mcp@0.2.4. It reads these environment variables: CRAWL_READINESS_API_KEY (secret) and CRAWL_READINESS_API_BASE; set them in the client's configuration for this server.

In the mcpServers JSON format that many desktop and editor MCP clients read, the entry looks like this (placeholders in angle brackets):

{
  "mcpServers": {
    "crawl-readiness": {
      "command": "npx",
      "args": [
        "-y",
        "crawl-readiness-mcp@0.2.4"
      ],
      "env": {
        "CRAWL_READINESS_API_KEY": "<secret>",
        "CRAWL_READINESS_API_BASE": "<value>"
      }
    }
  }
}

Derived from the registry entry, not tested here. What the server does, and on what terms, is set by its publisher; check its repository or website before giving it access to your accounts or files. How to add an MCP server to an assistant · Before you connect

More data servers

All 4,395 →
ServerRunsEndpoint or package
CraloPay-per-use tool marketplace for AI agents. Search, price-check, and call APIs via MCP.Remote · HTTPmcp.cralo.ai
CrashStory MCPColorado personal injury attorney directory, AI-analyzed review intelligence, and crash data.Remote · HTTP, SSEcrashstory-mcp-production.up.railway.app
Crawdar Business ResearchEvidence-backed public business research for agents with compact, pageable output.Remote · HTTPcrawdar.com
Crawl CensusAsk before you fetch: will this domain serve your crawler, refuse it, or charge it?Remote · HTTPcrawlcensus.com
Crawl4AgentCrawl4ai-compatible web crawler API: POST a URL, get clean LLM-ready Markdown with fit_markdown...Remote · HTTPcrawl4agent.saastemly.com
CrawlbergScrape, crawl, and map websites to Markdown or JSON via local CLI.Local · stdioOCI image (Docker): ghcr.io/xberg-io/crawlberg:1.9.0
CrawlbitFour free tools to check if AI engines can find, read and recognise a site. No API key.Local · stdionpm: crawlbit-mcp
crawlbrulee MCPEU-native web scraping for AI agents. scrape pages, map sites, run background jobs, check usage.Local · stdionpm: @crawlbrulee/mcp