TidyToolsAI Crawler Index · data as of · updated daily

AI crawler · Google · AI search

Which websites block Google-CloudVertexBot?

As of 2026-10-01, 61 of the 853 popular websites whose robots.txt we could read (7.2%) block Google-CloudVertexBot, Google's AI search crawler, on the home page; 9 more block it on some paths. Among the top 100 sites: 16.9%.

7.2%
of 853 readable sites block Google-CloudVertexBot
16.9%
of the top 100
#18
most blocked of 29 AI crawlers
783
sites allow it on the home page

Blocked by category

CategorySites readBlock Google-CloudVertexBot
Social4820.8%
News & media12616.7%
Travel1315.4%
Reference439.3%
Entertainment687.4%
Other2606.2%
E-commerce583.4%
Tech1540.6%
AI90%
Education200%
Finance160%
Government330%
Telecom50%

Popular sites that block Google-CloudVertexBot (61)

  1. facebook.com Social
  2. instagram.com Social
  3. twitter.com Social
  4. linkedin.com Social
  5. amazon.com E-commerce
  6. netflix.com Entertainment
  7. x.com Social
  8. pinterest.com Social
  9. reddit.com Social
  10. amazonvideo.com Entertainment
  11. nytimes.com News & media
  12. flickr.com Other
  13. theguardian.com News & media
  14. bbc.com News & media
  15. meraki.com Tech
  16. imdb.com Reference
  17. launchpad.net Other
  18. reuters.com News & media
  19. weibo.com Social
  20. weather.com News & media
  21. trustpilot.com Other
  22. bloomberg.com News & media
  23. wired.com News & media
  24. usatoday.com News & media
  25. tripadvisor.com Travel
  26. threads.com Social
  27. telegraph.co.uk News & media
  28. primevideo.com Entertainment
  29. dailymail.com News & media
  30. lemonde.fr News & media
  31. fb.com Other
  32. patreon.com Social
  33. disneyplus.com Entertainment
  34. people.com News & media
  35. academia.edu Reference
  36. theverge.com News & media
  37. corriere.it News & media
  38. investopedia.com News & media
  39. bfmtv.com News & media
  40. theconversation.com News & media
  41. gazzetta.it News & media
  42. amap.com Travel
  43. emag.ro E-commerce
  44. themoviedb.org Reference
  45. mydramalist.com Reference
  46. iltalehti.fi News & media
  47. tjk.org Other
  48. aka.ms Other
  49. sportfilm800.com News & media
  50. twkan.com Other
  51. rocketmoney.dev Other
  52. qcloud.com Other
  53. newyorker.com Other
  54. blackhub.team Other
  55. arstechnica.com News & media
  56. adrta.com Other
  57. hulu.com Entertainment
  58. lnkd.in Other
  59. sogou.com Other
  60. salesforceliveagent.com Other
  61. sedoparking.com Other

Sites that block Google-CloudVertexBot on some paths (9)

  1. hostinger.com Tech
  2. tradingview.com News & media
  3. wsj.com News & media
  4. indeed.com Other
  5. sohu.com Tech
  6. statista.com News & media
  7. jimdo.com Tech
  8. marketwatch.com E-commerce
  9. merkur.de Other

Trend

The daily series for Google-CloudVertexBot starts on 2026-10-01; a trend appears here as days accumulate.

Latest changes

No site changed its Google-CloudVertexBot rule in the latest daily run.

Block or allow Google-CloudVertexBot

User-agent: Google-CloudVertexBot
Disallow: /

Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.

FAQ

How many websites block Google-CloudVertexBot?
As of 2026-10-01, 61 of the 853 popular websites whose robots.txt we could read (7.2%) block Google-CloudVertexBot, Google's AI search crawler, on the home page; 9 more block it on some paths. Among the top 100 sites: 16.9%.
What is Google-CloudVertexBot?
Google-CloudVertexBot is Google's AI search crawler: it indexes pages so an AI search product can find and cite them. It is the #18 most blocked of the 29 AI crawlers in the index.
How do I block Google-CloudVertexBot in robots.txt?
Add these lines to https://<your-site>/robots.txt: "User-agent: Google-CloudVertexBot" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
How do I check whether my site blocks Google-CloudVertexBot?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of Google-CloudVertexBot and 28 other AI crawlers with the exact robots.txt rule.

Other AI crawlers

Also from Google: Google-Extended (14%), Google-GeminiNotebook (4.3%), Google-Agent (4.5%).