ActableSite AI Crawler Monitor
FreeNot checkedCheck AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
About
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
README
A remote, read-only MCP server for public website evidence, AI crawler policy, and explicit Crawler Watch, report, and Practice Radar offer metadata.
Tools
It exposes five tools over Streamable HTTP:
audit_public_websitechecks public identity, offer evidence, action paths, discovery files, metadata, and structured data.check_ai_crawler_policyevaluates eight OpenAI, Anthropic, Perplexity, and Google crawler tokens.get_crawler_watch_offerreturns the $9 monthly price, one-site scope, 15-minute cadence, confirmation rule, delivery, cancellation, and synthetic-check limitations. It cannot open checkout or purchase anything.get_full_report_offerreturns explicit metadata for an optional $19 one-time report. It cannot open checkout or purchase anything.get_practice_radar_offerreturns the current weekly-edition receipt, public sample, $39 monthly price, delivery, cancellation, and material NPI limitations. It cannot open checkout or purchase anything.
Connect
Claude Code:
claude mcp add --transport http actablesite https://actablesite.com/mcp
For another MCP client, add a remote server named actablesite with:
- URL:
https://actablesite.com/mcp - Transport: Streamable HTTP
- Authentication: none
For clients that require a local stdio command, clone this repository and run the dependency-free bridge:
git clone https://github.com/unitedideas/actablesite-mcp.git
cd actablesite-mcp
npm run start:stdio
The bridge contains the visible MCP implementation source in src/stdio.mjs. Distribution version 1.4.2 exposes the same five read-only tools and sends requests only to ActableSite's bounded public JSON APIs. It has no file, shell, browser, account, or write capability.
The same stdio process is available as a public, non-root container:
docker run --rm -i ghcr.io/unitedideas/actablesite-mcp:1.4.2
The hosted server implements MCP protocol version 2025-06-18 and is published in the official registry as com.actablesite/ai-crawler-robots-monitor version 1.4.2.
Discovery metadata
Directory scanners can read either machine-readable discovery surface:
- /.well-known/mcp.json identifies the official registry name, repository, remote endpoint, transport, authentication mode, and tool names.
- /.well-known/mcp/server-card.json provides static tool descriptions and JSON input schemas for scanners that cannot complete a live MCP handshake.
Both files describe the same unauthenticated endpoint and five-tool inventory verified by this repository.
Ongoing crawler monitoring
The MCP tools report current public evidence and do not create background jobs. Crawler Watch is the separate $9/month service for one public website: it checks crawler policy, synthetic homepage responses, sitemap.xml, and llms.txt every 15 minutes and emails after a changed state appears twice.
Its synthetic requests do not authenticate provider IP ranges or prove crawling, indexing, citation, ranking, or traffic.
Safety contract
- Public HTTP and HTTPS websites only; private, local, loopback, and reserved network destinations are rejected.
- No browser control, file access, account access, write, or mutation tools.
- No authentication header or API key is requested.
- Tool calls are limited to 60 per caller per hour.
- All three offer tools are informational and require explicit user confirmation before any separate checkout or purchase action.
- Practice Radar records do not prove licensure, active operation, demand, or buying intent and must be verified before contact.
- Results do not guarantee crawling, indexing, citation, ranking, recommendation, traffic, legal compliance, or revenue.
Verify the live server
Node.js 20 or newer is required.
npm test
npm run verify:live
The verifier initializes the live server, checks its protocol and version, and confirms the exact read-only tool inventory. It does not call an audit tool or create a funnel event.
Repository scope
This repository contains a runnable stdio implementation, public registry manifest, connection contract, live verifier, and security policy for the hosted ActableSite MCP server. The production runtime is operated at https://actablesite.com/mcp.
License
MIT
Installing ActableSite AI Crawler Monitor
This server has no published package — it is built from source. Open the repository and follow its README.
▸ github.com/unitedideas/actablesite-mcpFAQ
Is ActableSite AI Crawler Monitor MCP free?
Yes, ActableSite AI Crawler Monitor MCP is free — one-click install via Unyly at no cost.
Does ActableSite AI Crawler Monitor need an API key?
No, ActableSite AI Crawler Monitor runs without API keys or environment variables.
Is ActableSite AI Crawler Monitor hosted or self-hosted?
A hosted option is available: Unyly runs the server in the cloud, no local setup required.
How do I install ActableSite AI Crawler Monitor in Claude Desktop, Claude Code or Cursor?
Open ActableSite AI Crawler Monitor on unyly.org, pick your client tab (Claude Desktop, Claude Code, Cursor) and press Install — the config is generated automatically, no JSON editing.
Related MCPs
Fetch
Web content fetching and conversion for efficient LLM usage.
AWS KB Retrieval
Retrieval from AWS Knowledge Base using Bedrock Agent Runtime.
by modelcontextprotocolSpring AI MCP Server
Provides auto-configuration for setting up an MCP server in Spring Boot applications.
llm-analysis-assistant
A very streamlined mcp client that supports calling and monitoring stdio/sse/streamableHttp, and can also view request responses through the /logs page. It also
by xuzexin-hzCompare ActableSite AI Crawler Monitor with
Not sure what to pick?
Find your stack in 60 seconds
Author?
Embed badge for your README
Browse similar
All ai MCPs
