TidyToolsAI Crawler Index · data as of · updated daily

AI crawler · Perplexity · User-triggered fetch

Which websites block Perplexity-User?

As of 2026-11-16, 506 of the 8,313 popular websites whose robots.txt we could read (6.1%) block Perplexity-User, Perplexity's user-triggered AI fetcher, on the home page; 123 more block it on some paths. Among the top 100 sites: 16.9%.

6.1%
of 8,313 readable sites block Perplexity-User
16.9%
of the top 100
#15
most blocked of 29 AI crawlers
7,684
sites allow it on the home page

Blocked by category

CategorySites readBlock Perplexity-User
Social5121.6%
News & media39219.9%
Travel137.7%
Reference437%
Other66515.7%
E-commerce1164.3%
Finance1023.9%
Entertainment3323.6%
Government2033.4%
Education2321.7%
AI100%
Tech1620%
Telecom60%

Popular sites that block Perplexity-User (506)

Showing the 200 most popular of 506. The full list is in sites.csv.

Sites that block Perplexity-User on some paths (123)

Showing the 60 most popular of 123. The full list is in sites.csv.

Trend

The daily series for Perplexity-User starts on 2026-11-16; a trend appears here as days accumulate.

Latest changes

Block or allow Perplexity-User

User-agent: Perplexity-User
Disallow: /

Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.

FAQ

How many websites block Perplexity-User?
As of 2026-11-16, 506 of the 8,313 popular websites whose robots.txt we could read (6.1%) block Perplexity-User, Perplexity's user-triggered AI fetcher, on the home page; 123 more block it on some paths. Among the top 100 sites: 16.9%.
What is Perplexity-User?
Perplexity-User is Perplexity's user-triggered AI fetcher: it fetches a page when a user asks the assistant about it. It is the #15 most blocked of the 29 AI crawlers in the index.
How do I block Perplexity-User in robots.txt?
Add these lines to https://<your-site>/robots.txt: "User-agent: Perplexity-User" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
How do I check whether my site blocks Perplexity-User?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of Perplexity-User and 28 other AI crawlers with the exact robots.txt rule.

Other AI crawlers

Also from Perplexity: PerplexityBot (8.2%).