scrapy-mcp
A headless web-scraping MCP server built on Scrapy, providing tools for polite fetching, CSS/XPath extraction, link/table extraction, sitemap and robots.txt reading, and bounded asynchronous crawls.
浏览器自动化
0 个平台
0 项功能
安装
uvx scrapy-mcp
把上面的配置写入 MCP 客户端的配置文件（Claude Desktop 为 claude_desktop_config.json），重启客户端后生效。
{
"mcpServers": {
"scrapy-mcp-scraper": {
"command": "uvx",
"args": [
"scrapy-mcp"
],
"env": {
"SCRAPY_MCP_JOB_DIR": "<SCRAPY_MCP_JOB_DIR>",
"SCRAPY_MCP_LOG_LEVEL": "<SCRAPY_MCP_LOG_LEVEL>",
"SCRAPY_MCP_MAX_BYTES": "<SCRAPY_MCP_MAX_BYTES>",
"SCRAPY_MCP_USER_AGENT": "<SCRAPY_MCP_USER_AGENT>",
"SCRAPY_MCP_OBEY_ROBOTS": "<SCRAPY_MCP_OBEY_ROBOTS>",
"SCRAPY_MCP_RETRY_TIMES": "<SCRAPY_MCP_RETRY_TIMES>",
"SCRAPY_MCP_AUTOTHROTTLE": "<SCRAPY_MCP_AUTOTHROTTLE>",
"SCRAPY_MCP_JOB_TTL_DAYS": "<SCRAPY_MCP_JOB_TTL_DAYS>"
}
}
}
}把上面的配置写入 MCP 客户端的配置文件（Claude Desktop 为 claude_desktop_config.json），重启客户端后生效。
数据来源：公开的 MCP Server 目录。本站为独立第三方目录，与各 MCP Server 维护方均无隶属或背书关系；名称与简介如实标注，介绍文案由本站再加工。
