Scrapy MCP Server
A powerful web scraping MCP server built on Scrapy and FastMCP that supports multiple scraping methods (HTTP, Scrapy, browser automation), anti-detection techniques, form handling, and concurrent crawling. Designed for commercial environments with enterprise-grade features like intelligent retry mechanisms, performance monitoring, and configurable data extraction.
Install
uvx negentropy-perceives
Paste the configuration above into your MCP client config (claude_desktop_config.json for Claude Desktop) and restart the client.
{
"mcpServers": {
"negentropy-perceives": {
"command": "uvx",
"args": [
"negentropy-perceives"
],
"env": {
"SCRAPY_MCP_PROXY_URL": "<SCRAPY_MCP_PROXY_URL>",
"SCRAPY_MCP_USE_PROXY": "<SCRAPY_MCP_USE_PROXY>",
"SCRAPY_DOWNLOAD_DELAY": "<SCRAPY_DOWNLOAD_DELAY>",
"SCRAPY_MCP_MAX_RETRIES": "<SCRAPY_MCP_MAX_RETRIES>",
"SCRAPY_MCP_SERVER_NAME": "<SCRAPY_MCP_SERVER_NAME>",
"SCRAPY_MCP_SERVER_VERSION": "<SCRAPY_MCP_SERVER_VERSION>",
"SCRAPY_CONCURRENT_REQUESTS": "<SCRAPY_CONCURRENT_REQUESTS>",
"SCRAPY_MCP_BROWSER_TIMEOUT": "<SCRAPY_MCP_BROWSER_TIMEOUT>"
}
}
}
}Paste the configuration above into your MCP client config (claude_desktop_config.json for Claude Desktop) and restart the client.
Sources: public MCP Server directories. This is an independent third-party directory with no affiliation to or endorsement from the maintainers of the listed servers.
