AI crawler · Apple · AI search
Which websites block Applebot?
As of 2026-10-01, 22 of the 853 popular websites whose robots.txt we could read (2.6%) block Applebot, Apple's AI search crawler, on the home page; 100 more block it on some paths. Among the top 100 sites: 1.2%.
2.6%
of 853 readable sites block Applebot
1.2%
of the top 100
#29
most blocked of 29 AI crawlers
731
sites allow it on the home page
Blocked by category
| Category | Sites read | Block Applebot |
|---|---|---|
| News & media | 126 | 7.9% |
| E-commerce | 58 | 3.4% |
| Government | 33 | 3% |
| Other | 260 | 2.7% |
| Reference | 43 | 2.3% |
| Social | 48 | 2.1% |
| AI | 9 | 0% |
| Education | 20 | 0% |
| Entertainment | 68 | 0% |
| Finance | 16 | 0% |
| Tech | 154 | 0% |
| Telecom | 5 | 0% |
| Travel | 13 | 0% |
Popular sites that block Applebot (22)
- reddit.com Social
- uol.com.br News & media
- usatoday.com News & media
- cnet.com News & media
- repubblica.it News & media
- corriere.it News & media
- theconversation.com News & media
- emag.ro E-commerce
- themoviedb.org Reference
- sapo.pt News & media
- dw.com News & media
- aka.ms Other
- sportfilm800.com News & media
- ad-contents.jp Other
- rocketmoney.dev Other
- qcloud.com Other
- blackhub.team Other
- zdnet.com News & media
- adrta.com Other
- disdarrummers.shop E-commerce
- meb.gov.tr Government
- ign.com Other
Sites that block Applebot on some paths (100)
- facebook.com Social
- instagram.com Social
- mail.ru Tech
- twitter.com Social
- linkedin.com Social
- netflix.com Entertainment
- x.com Social
- pinterest.com Social
- adobe.com Tech
- samsung.com Tech
- snapchat.com Social
- vk.com Social
- nytimes.com News & media
- vk.ru Social
- flickr.com Other
- ebay.com E-commerce
- oracle.com Tech
- android.com Tech
- cisco.com Tech
- meraki.com Tech
- imdb.com Reference
- booking.com Travel
- reuters.com News & media
- alibaba.com E-commerce
- drom.ru Other
- trustpilot.com Other
- wsj.com News & media
- indeed.com Other
- aliexpress.com E-commerce
- dailymotion.com Entertainment
- temu.com E-commerce
- arxiv.org Reference
- sohu.com Tech
- cnbc.com News & media
- quora.com Reference
- gosuslugi.ru Other
- capcut.com Other
- trendmicro.com Tech
- rakuten.co.jp E-commerce
- time.com News & media
- tripadvisor.com Travel
- threads.com Social
- avito.ru E-commerce
- linode.com Tech
- shopee.com.br E-commerce
- perplexity.ai AI
- airbnb.com Travel
- fb.com Other
- ya.ru Tech
- bestbuy.com E-commerce
- ameblo.jp Social
- disneyplus.com Entertainment
- deepl.com AI
- agoda.com Travel
- att.com Telecom
- academia.edu Reference
- pikabu.ru Social
- binance.com Finance
- note.com Social
- ameba.jp Social
Showing the 60 most popular of 100. The full list is in sites.csv.
Trend
The daily series for Applebot starts on 2026-10-01; a trend appears here as days accumulate.
Latest changes
- dailymotion.com: blocked → partial
- abc.net.au: allowed → partial
Block or allow Applebot
User-agent: Applebot Disallow: /
Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.
FAQ
- How many websites block Applebot?
- As of 2026-10-01, 22 of the 853 popular websites whose robots.txt we could read (2.6%) block Applebot, Apple's AI search crawler, on the home page; 100 more block it on some paths. Among the top 100 sites: 1.2%.
- What is Applebot?
- Applebot is Apple's AI search crawler: it indexes pages so an AI search product can find and cite them. It is the #29 most blocked of the 29 AI crawlers in the index.
- How do I block Applebot in robots.txt?
- Add these lines to https://<your-site>/robots.txt: "User-agent: Applebot" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
- How do I check whether my site blocks Applebot?
- Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of Applebot and 28 other AI crawlers with the exact robots.txt rule.
Other AI crawlers
Also from Apple: Applebot-Extended (14.3%).
- CCBot 19.5%
- Bytespider 17.7%
- GPTBot 17%
- ClaudeBot 16.3%
- Meta-ExternalAgent 14.4%
- Applebot-Extended 14.3%
- Google-Extended 14%
- cohere-ai 13.7%
- Amazonbot 13.1%
- PerplexityBot 12.7%
- ChatGPT-User 9.6%
- Perplexity-User 8.9%
- meta-externalfetcher 8.7%
- MistralAI-User 8.4%
- Claude-User 8.3%
- Claude-SearchBot 8.1%
- DuckAssistBot 8.1%
- Google-CloudVertexBot 7.2%
- OAI-SearchBot 6.7%
- meta-webindexer 6.3%
- Amzn-SearchBot 5.9%
- Amzn-User 5.5%
- MistralAI-Index 4.7%
- MistralAI-Training 4.7%
- Google-Agent 4.5%
- OAI-AdsBot 4.3%
- Google-GeminiNotebook 4.3%
- meta-externalads 3.8%