TidyToolsAI Crawler Index · data as of · updated daily

AI crawler · Cohere · Training

Which websites block cohere-ai?

As of 2026-11-16, 1,052 of the 8,313 popular websites whose robots.txt we could read (12.7%) block cohere-ai, Cohere's AI training crawler, on the home page; 67 more block it on some paths. Among the top 100 sites: 19.3%.

12.7%
of 8,313 readable sites block cohere-ai
19.3%
of the top 100
#9
most blocked of 29 AI crawlers
7,194
sites allow it on the home page

Blocked by category

CategorySites readBlock cohere-ai
News & media39229.3%
Social5125.5%
Entertainment33219%
Reference4314%
Other665112.3%
E-commerce11611.2%
Travel137.7%
Finance1026.9%
Education2323.9%
Government2032.5%
Tech1621.9%
AI100%
Telecom60%

Popular sites that block cohere-ai (1,052)

Showing the 200 most popular of 1,052. The full list is in sites.csv.

Sites that block cohere-ai on some paths (67)

Showing the 60 most popular of 67. The full list is in sites.csv.

Trend

The daily series for cohere-ai starts on 2026-11-16; a trend appears here as days accumulate.

Latest changes

Block or allow cohere-ai

User-agent: cohere-ai
Disallow: /

Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.

FAQ

How many websites block cohere-ai?
As of 2026-11-16, 1,052 of the 8,313 popular websites whose robots.txt we could read (12.7%) block cohere-ai, Cohere's AI training crawler, on the home page; 67 more block it on some paths. Among the top 100 sites: 19.3%.
What is cohere-ai?
cohere-ai is Cohere's AI training crawler: it collects pages to train AI models. It is the #9 most blocked of the 29 AI crawlers in the index.
How do I block cohere-ai in robots.txt?
Add these lines to https://<your-site>/robots.txt: "User-agent: cohere-ai" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
How do I check whether my site blocks cohere-ai?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of cohere-ai and 28 other AI crawlers with the exact robots.txt rule.

Other AI crawlers