TidyToolsAI Crawler Index · data as of · updated daily

AI crawler · Google · AI search

Which websites block Google-CloudVertexBot?

As of 2026-11-16, 733 of the 8,313 popular websites whose robots.txt we could read (8.8%) block Google-CloudVertexBot, Google's AI search crawler, on the home page; 27 more block it on some paths. Among the top 100 sites: 16.9%.

8.8%
of 8,313 readable sites block Google-CloudVertexBot
16.9%
of the top 100
#10
most blocked of 29 AI crawlers
7,553
sites allow it on the home page

Blocked by category

CategorySites readBlock Google-CloudVertexBot
Social5123.5%
Travel1315.4%
Entertainment33213.9%
News & media39212.8%
Reference439.3%
Other66518.9%
E-commerce1167.8%
Finance1024.9%
Education2323.9%
Government2032%
AI100%
Tech1620%
Telecom60%

Popular sites that block Google-CloudVertexBot (733)

Showing the 200 most popular of 733. The full list is in sites.csv.

Sites that block Google-CloudVertexBot on some paths (27)

Trend

The daily series for Google-CloudVertexBot starts on 2026-11-16; a trend appears here as days accumulate.

Latest changes

Block or allow Google-CloudVertexBot

User-agent: Google-CloudVertexBot
Disallow: /

Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.

FAQ

How many websites block Google-CloudVertexBot?
As of 2026-11-16, 733 of the 8,313 popular websites whose robots.txt we could read (8.8%) block Google-CloudVertexBot, Google's AI search crawler, on the home page; 27 more block it on some paths. Among the top 100 sites: 16.9%.
What is Google-CloudVertexBot?
Google-CloudVertexBot is Google's AI search crawler: it indexes pages so an AI search product can find and cite them. It is the #10 most blocked of the 29 AI crawlers in the index.
How do I block Google-CloudVertexBot in robots.txt?
Add these lines to https://<your-site>/robots.txt: "User-agent: Google-CloudVertexBot" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
How do I check whether my site blocks Google-CloudVertexBot?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of Google-CloudVertexBot and 28 other AI crawlers with the exact robots.txt rule.

Other AI crawlers

Also from Google: Google-Extended (13.2%), Google-GeminiNotebook (3.3%), Google-Agent (3.6%).