GPTBot is OpenAI's AI training crawler: it collects pages to train AI models. It is the #3 most blocked of the 29 AI crawlers in the index.

How do I block GPTBot in robots.txt?

Add these lines to https:// /robots.txt: "User-agent: GPTBot" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.

How do I check whether my site blocks GPTBot?

Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of GPTBot and 28 other AI crawlers with the exact robots.txt rule.

AI crawler · OpenAI · Training

Which websites block GPTBot?

As of 2026-10-01, 145 of the 853 popular websites whose robots.txt we could read (17%) block GPTBot, OpenAI's AI training crawler, on the home page; 47 more block it on some paths. Among the top 100 sites: 21.7%.

17%

of 853 readable sites block GPTBot

21.7%

of the top 100

most blocked of 29 AI crawlers

661

sites allow it on the home page

Blocked by category

Category	Sites read	Block GPTBot
News & media	126	43.7%
Social	48	33.3%
Reference	43	27.9%
Travel	13	15.4%
Other	260	13.5%
Entertainment	68	13.2%
AI	9	11.1%
E-commerce	58	10.3%
Government	33	6.1%
Tech	154	4.5%
Education	20	0%
Finance	16	0%
Telecom	5	0%

Trend

The daily series for GPTBot starts on 2026-10-01; a trend appears here as days accumulate.

Latest changes

dailymotion.com: blocked → allowed
eff.org: allowed → partial

Block or allow GPTBot

User-agent: GPTBot
Disallow: /

Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.

FAQ

How many websites block GPTBot?: As of 2026-10-01, 145 of the 853 popular websites whose robots.txt we could read (17%) block GPTBot, OpenAI's AI training crawler, on the home page; 47 more block it on some paths. Among the top 100 sites: 21.7%.
What is GPTBot?: GPTBot is OpenAI's AI training crawler: it collects pages to train AI models. It is the #3 most blocked of the 29 AI crawlers in the index.
How do I block GPTBot in robots.txt?: Add these lines to https://<your-site>/robots.txt: "User-agent: GPTBot" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
How do I check whether my site blocks GPTBot?: Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of GPTBot and 28 other AI crawlers with the exact robots.txt rule.

Other AI crawlers

Also from OpenAI: OAI-SearchBot (6.7%), ChatGPT-User (9.6%), OAI-AdsBot (4.3%).