MCP Server Directory

Finds the robots.txt lines that reserve nothing — retired crawler tokens, shadowed groups, rules abo

AI crawler checker: shows which page content GPTBot and ClaudeBot can't read without JavaScript.

Self-hosted, Tavily-compatible web search, extract, crawl and map tools for AI agents.

ExtractEdge io.github.vasilicasijarvis/extractedge

Any web page -> clean markdown. Firecrawl-lite, 0.001 USDC/extract (x402, Base), free demo.

Scrapingbee io.github.pipeworx-io/scrapingbee

ScrapingBee MCP — wraps the ScrapingBee headless-scraping API (scrapingbee.com)

Ashby Jobs Scraper & API - All Companies, Remote, Last 24 Hours io.github.AnshumanAtrey/ashby-jobs-search

Ashby Jobs Scraper - Search Every Ashby Job Board

url-to-markdown io.github.DataLeadsPRO/url-to-markdown

Convert any webpage to clean LLM-ready markdown, extraction-first, with article and news modes.

oassis — web scraping, crawling and browser control for AI agents dev.oassis/scraper-crawler

Web Tools and Web Tasks for AI agents. 100-credit MCP trial; API-key credits or x402.

Scans RAG content/scraped pages for indirect prompt injection

VidTL Video Editor com.vidtl/video-editor

Edit video in your open VidTL browser editor: cuts, subtitles, audio cleanup, effects, export.

Found17 com.found17/found17

Public web tools for agents: product extraction, claim checks, webpage QA and ranked audits.

Capture web pages as screenshots (returned inline) or PDFs via the Screenshotty API.

Unstuck com.unstuckapi/unstuck

Web search, read any page, verify or find emails, screenshots, keyword data. One key, pay per call.

Agentic browser harness: your agent CLI in your real, signed-in browser, from a side panel or MCP.

MCP server for Island Browser's Admin Actions, Audit, Compromised Credential, and Device APIs.

MCP server for browser and desktop automation: OCR, image recognition, real mouse and keyboard input

Citewren dev.workers.cybermax-tools.cybermax/citewren

AI search readiness MCP: score a site for ChatGPT/Claude/Perplexity crawlers. 3 free/day.

AI browser agent: browses, clicks, types, and reads real web pages from plain-English instructions.

DeepSFetch io.github.ermultimidia/deepsfetch

Web scraping, extraction and search for AI agents. Hosted MCP with workspace API keys.

Contact Details Scraper io.github.saulius876-lgtm/contact-details

Emails, phones and social links from any list of websites. Pay only for sites with a contact.

MeshArc dev.mesharc/mesharc

Crawl and scrape websites to clean markdown, and track what changed between crawls.

LexLint: compliance lint for AI, scraping, and privacy law io.lexlint/lexlint

Compliance lint for AI, scraping, and privacy law. Cited findings in 200 or more jurisdictions.

AIReady io.github.everyai-com/ai-ready

Scan and fix your site's AI discoverability: crawler access, llms.txt, JSON-LD. Free.

Naukri Jobs Scraper io.github.saulius876-lgtm/naukri-jobs

Search Naukri.com jobs in India: titles, companies, salaries, skills, and which companies hire most.