TidyToolsAI Crawler Index · data as of · updated daily

AI Crawler Index · Anthropic · ClaudeBot

Which .ai websites block ClaudeBot?

As of 2026-11-16, 3 of the 61 .ai websites whose robots.txt we could read (4.9%) block ClaudeBot, Anthropic's AI training crawler, on the home page, compared with 14.7% of all sites; 3 more block it on some paths.

4.9%
of 61 .ai websites block ClaudeBot
14.7%
of all sites
3
sites block it

All 3 .ai websites that block ClaudeBot

#SitePolicyCrawlers blocked
5050loudecho.aiBlocks some AI search11 of 29
6143scite.aiBlocks training only2 of 29
6258mgln.aiBlocks all AI search29 of 29

Popular .ai websites that allow ClaudeBot

Other AI crawlers and .ai websites

ClaudeBot in other segments

ClaudeBot overview · All .ai websites

FAQ

Which .ai websites block ClaudeBot?
loudecho.ai, scite.ai, mgln.ai.
How many .ai websites block ClaudeBot?
As of 2026-11-16, 3 of the 61 .ai websites whose robots.txt we could read (4.9%) block ClaudeBot, Anthropic's AI training crawler, on the home page, compared with 14.7% of all sites; 3 more block it on some paths.
How do I block or allow ClaudeBot on my site?
Add "User-agent: ClaudeBot" followed by "Disallow: /" to https://<your-site>/robots.txt to block it, or "Allow: /" to allow it. Check the result with the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker.