TidyToolsAI Crawler Index · data as of · updated daily

AI crawler · OpenAI · User-triggered fetch

Which websites block ChatGPT-User?

As of 2026-11-16, 616 of the 8,313 popular websites whose robots.txt we could read (7.4%) block ChatGPT-User, OpenAI's user-triggered AI fetcher, on the home page; 254 more block it on some paths. Among the top 100 sites: 15.7%.

7.4%
of 8,313 readable sites block ChatGPT-User
15.7%
of the top 100
#12
most blocked of 29 AI crawlers
7,443
sites allow it on the home page

Blocked by category

CategorySites readBlock ChatGPT-User
Social5123.5%
News & media39220.9%
Reference4311.6%
AI1010%
Other66517.2%
Entertainment3324.5%
Government2033.9%
E-commerce1163.4%
Finance1022.9%
Education2321.7%
Tech1620.6%
Telecom60%
Travel130%

Popular sites that block ChatGPT-User (616)

Showing the 200 most popular of 616. The full list is in sites.csv.

Sites that block ChatGPT-User on some paths (254)

Showing the 60 most popular of 254. The full list is in sites.csv.

Trend

DateShare blocking ChatGPT-User
2026-11-167.4%
2026-11-157.3%
2026-11-147.3%
2026-11-137.3%
2026-11-127.3%
2026-11-117.4%
2026-11-107.4%
2026-11-097.4%
2026-11-087.4%
2026-11-077.4%
2026-11-067.5%
2026-11-057.4%
2026-11-047.4%
2026-11-037.4%
2026-11-027.4%
2026-11-017.4%
2026-10-317.4%
2026-10-307.4%
2026-10-297.4%
2026-10-287.4%
2026-10-277.4%
2026-10-267.4%
2026-10-257.5%
2026-10-247.4%
2026-10-237.4%
2026-10-227.6%
2026-10-217.9%
2026-10-207.4%
2026-10-197.3%
2026-10-187.3%

Latest changes

Block or allow ChatGPT-User

User-agent: ChatGPT-User
Disallow: /

Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.

FAQ

How many websites block ChatGPT-User?
As of 2026-11-16, 616 of the 8,313 popular websites whose robots.txt we could read (7.4%) block ChatGPT-User, OpenAI's user-triggered AI fetcher, on the home page; 254 more block it on some paths. Among the top 100 sites: 15.7%.
What is ChatGPT-User?
ChatGPT-User is OpenAI's user-triggered AI fetcher: it fetches a page when a user asks the assistant about it. It is the #12 most blocked of the 29 AI crawlers in the index.
How do I block ChatGPT-User in robots.txt?
Add these lines to https://<your-site>/robots.txt: "User-agent: ChatGPT-User" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
How do I check whether my site blocks ChatGPT-User?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of ChatGPT-User and 28 other AI crawlers with the exact robots.txt rule.

Other AI crawlers

Also from OpenAI: GPTBot (15.4%), OAI-SearchBot (5.8%), OAI-AdsBot (3%).