TidyToolsAI Crawler Index · data as of · updated daily

AI crawler · Cohere · Training

Which websites block cohere-ai?

As of 2026-10-01, 117 of the 853 popular websites whose robots.txt we could read (13.7%) block cohere-ai, Cohere's AI training crawler, on the home page; 16 more block it on some paths. Among the top 100 sites: 19.3%.

13.7%
of 853 readable sites block cohere-ai
19.3%
of the top 100
#8
most blocked of 29 AI crawlers
720
sites allow it on the home page

Blocked by category

CategorySites readBlock cohere-ai
News & media12642.1%
Social4822.9%
Entertainment6816.2%
Reference4314%
Other26010.8%
Travel137.7%
E-commerce585.2%
Tech1542.6%
AI90%
Education200%
Finance160%
Government330%
Telecom50%

Popular sites that block cohere-ai (117)

  1. facebook.com Social
  2. instagram.com Social
  3. twitter.com Social
  4. linkedin.com Social
  5. amazon.com E-commerce
  6. netflix.com Entertainment
  7. x.com Social
  8. pinterest.com Social
  9. yahoo.com Tech
  10. reddit.com Social
  11. amazonvideo.com Entertainment
  12. snapchat.com Social
  13. nytimes.com News & media
  14. flickr.com Other
  15. forbes.com News & media
  16. bbc.com News & media
  17. meraki.com Tech
  18. imdb.com Reference
  19. launchpad.net Other
  20. reuters.com News & media
  21. nature.com Reference
  22. weibo.com Social
  23. ea.com Entertainment
  24. trustpilot.com Other
  25. bloomberg.com News & media
  26. wsj.com News & media
  27. aol.com Tech
  28. cnbc.com News & media
  29. ft.com News & media
  30. wired.com News & media
  31. uol.com.br News & media
  32. usatoday.com News & media
  33. tripadvisor.com Travel
  34. threads.com Social
  35. telegraph.co.uk News & media
  36. nintendo.com Entertainment
  37. techcrunch.com News & media
  38. sagepub.com Reference
  39. primevideo.com Entertainment
  40. dailymail.com News & media
  41. lemonde.fr News & media
  42. fb.com Other
  43. cnet.com News & media
  44. bild.de News & media
  45. apnews.com News & media
  46. ted.com Entertainment
  47. welt.de News & media
  48. disneyplus.com Entertainment
  49. people.com News & media
  50. academia.edu Reference
  51. theverge.com News & media
  52. figma.com Tech
  53. nbcnews.com News & media
  54. onet.pl News & media
  55. repubblica.it News & media
  56. investopedia.com News & media
  57. pixiv.net Entertainment
  58. flashscore.com News & media
  59. theatlantic.com News & media
  60. bfmtv.com News & media
  61. huffpost.com News & media
  62. cookpad.com Entertainment
  63. aftonbladet.se News & media
  64. nypost.com News & media
  65. theconversation.com News & media
  66. ad.nl News & media
  67. nu.nl News & media
  68. elconfidencial.com News & media
  69. vg.no News & media
  70. spiegel.de News & media
  71. dcinside.com Social
  72. hln.be News & media
  73. emag.ro E-commerce
  74. abc.net.au News & media
  75. themoviedb.org Reference
  76. ladepeche.fr News & media
  77. mydramalist.com Reference
  78. actu.fr News & media
  79. nikkansports.com News & media
  80. huffingtonpost.com News & media
  81. dagbladet.no News & media
  82. ilmeteo.it News & media
  83. sapo.pt News & media
  84. dw.com News & media
  85. iltalehti.fi News & media
  86. aljazeera.com News & media
  87. tjk.org Other
  88. buzzfeed.com Other
  89. aka.ms Other
  90. sportfilm800.com News & media
  91. twkan.com Other
  92. rocketmoney.dev Other
  93. qcloud.com Other
  94. newsweek.com News & media
  95. newyorker.com Other
  96. blackhub.team Other
  97. arstechnica.com News & media
  98. zdnet.com News & media
  99. adrta.com Other
  100. marketwatch.com E-commerce
  101. scientificamerican.com Other
  102. merkur.de Other
  103. hulu.com Entertainment
  104. thelancet.com Other
  105. iqiyi.com Entertainment
  106. wikihow.com Other
  107. oreilly.com Other
  108. hola.com Other
  109. jamanetwork.com Other
  110. fastcompany.com Other
  111. liquipedia.net Other
  112. lnkd.in Other
  113. sogou.com Other
  114. ign.com Other
  115. gizmodo.com Other
  116. salesforceliveagent.com Other
  117. sedoparking.com Other

Sites that block cohere-ai on some paths (16)

  1. soundcloud.com Entertainment
  2. android.com Tech
  3. ubuntu.com Tech
  4. sohu.com Tech
  5. statista.com News & media
  6. trendmicro.com Tech
  7. squarespace.com Tech
  8. jotform.com Tech
  9. airbnb.com Travel
  10. ring.com Tech
  11. jimdo.com Tech
  12. fmkorea.com Social
  13. amap.com Travel
  14. deezer.com Other
  15. betpawa.ug Finance
  16. kargo.com Other

Trend

The daily series for cohere-ai starts on 2026-10-01; a trend appears here as days accumulate.

Latest changes

Block or allow cohere-ai

User-agent: cohere-ai
Disallow: /

Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.

FAQ

How many websites block cohere-ai?
As of 2026-10-01, 117 of the 853 popular websites whose robots.txt we could read (13.7%) block cohere-ai, Cohere's AI training crawler, on the home page; 16 more block it on some paths. Among the top 100 sites: 19.3%.
What is cohere-ai?
cohere-ai is Cohere's AI training crawler: it collects pages to train AI models. It is the #8 most blocked of the 29 AI crawlers in the index.
How do I block cohere-ai in robots.txt?
Add these lines to https://<your-site>/robots.txt: "User-agent: cohere-ai" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
How do I check whether my site blocks cohere-ai?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of cohere-ai and 28 other AI crawlers with the exact robots.txt rule.

Other AI crawlers