AI crawler · Apple · AI search
Which websites block Applebot?
As of 2026-11-16, 267 of the 8,313 popular websites whose robots.txt we could read (3.2%) block Applebot, Apple's AI search crawler, on the home page; 488 more block it on some paths. Among the top 100 sites: 1.2%.
3.2%
of 8,313 readable sites block Applebot
1.2%
of the top 100
#27
most blocked of 29 AI crawlers
7,558
sites allow it on the home page
Blocked by category
| Category | Sites read | Block Applebot |
|---|---|---|
| News & media | 392 | 6.9% |
| Entertainment | 332 | 3.3% |
| Other | 6651 | 3.3% |
| Finance | 102 | 2.9% |
| Social | 51 | 2% |
| E-commerce | 116 | 1.7% |
| Government | 203 | 1.5% |
| Education | 232 | 0.4% |
| AI | 10 | 0% |
| Reference | 43 | 0% |
| Tech | 162 | 0% |
| Telecom | 6 | 0% |
| Travel | 13 | 0% |
Popular sites that block Applebot (267)
- reddit.com Social
- uol.com.br News & media
- cnet.com News & media
- repubblica.it News & media
- corriere.it News & media
- theconversation.com News & media
- ad.nl News & media
- nu.nl News & media
- hln.be News & media
- sapo.pt News & media
- dw.com News & media
- aka.ms Other
- ad-contents.jp Other
- qcloud.com Other
- zdnet.com News & media
- rocketmoney.dev Other
- adrta.com Other
- disdarrummers.shop E-commerce
- blackhub.team Other
- ign.com Other
- mashable.com Other
- openssl.org Other
- constellix.com Other
- techradar.com Other
- kasserver.com Other
- co.com Other
- tappx.com Other
- expressapisv2.net News & media
- urldefense.com Other
- gutenberg.org Other
- livescience.com Other
- huntress.io Other
- system-monitor.com Other
- franceinfo.fr Other
- idnes.cz Other
- wa.gov Government
- jwpltx.com Other
- chicagotribune.com News & media
- hellotalk8.com Other
- wargaming.net Other
- tomshardware.com Other
- lifehacker.com Other
- dyz9.top Other
- lenovomm.com Other
- stuff.co.nz Other
- rs6.net Other
- onenote.com Other
- entrust.net Other
- nydailynews.com News & media
- sportradarserving.com News & media
- space.com Other
- backblazeb2.com Other
- findagrave.com Other
- wordreference.com Other
- tvtropes.org Other
- cnki.net Other
- kompas.com Other
- snopes.com Other
- myhostadmin.net Other
- helsinki.fi Other
- paycomonline.net Finance
- lastampa.it Other
- feinidi.com Other
- feednews.com News & media
- doh.pub Other
- ezoic.net Other
- tomsguide.com Other
- biorxiv.org Other
- dr.dk Other
- dayforcehcm.com Other
- ilsole24ore.com Other
- arte.tv Entertainment
- codeforces.com Other
- lapresse.ca News & media
- michelin.com Other
- lesechos.fr Other
- chinatimes.com News & media
- mercurynews.com News & media
- everydayhealth.com Other
- gentoo.org Other
- braintreegateway.com Other
- mypurecloud.com Other
- pro32connect.ru Other
- bustle.com Other
- francetvinfo.fr Other
- tv2.no Other
- afterpay.com Finance
- pcgamer.com Entertainment
- mlnba.fun Other
- jivosite.com Other
- ilmessaggero.it Other
- storygize.net Other
- diigo.com Other
- nimo.tv Entertainment
- securedvisit.com Other
- denverpost.com News & media
- megaphone.fm Other
- lunalabs.io Other
- parismatch.com Other
- gamesradar.com Entertainment
- emol.com Other
- carview.co.jp Other
- expediapartnercentral.com Other
- sandiegouniontribune.com News & media
- rp-online.de Other
- islonline.net Other
- nicematin.com Other
- dextrodedenda.top Other
- gelderlander.nl Other
- midasbuy.com E-commerce
- military.com Other
- athome.co.jp Other
- zol.com.cn Other
- yclients.com Other
- bhaskar.com Other
- quay.io Other
- baltimoresun.com Other
- hal.science Other
- destentor.nl Other
- mailin.fr Other
- volkskrant.nl Other
- seesaa.net Other
- healow.com Other
- spokeo.com Other
- servedbyadbutler.com Other
- leggo.it Other
- france.tv Entertainment
- eurogamer.net Entertainment
- ayhal.com Other
- mjt.lu Other
- nefisyemektarifleri.com Other
- hs-sites.com Other
- javbus.com Other
- ilmattino.it Other
- portalsamorzadowy.pl Other
- widen.net Other
- rynekzdrowia.pl Other
- futura-sciences.com Other
- 13o.net Other
- aruodas.lt Other
- wuytg.com Other
- manhuagui.com Other
- wfxs.tw Other
- rip.ie Other
- yangkeduo.com Other
- spiceworks.com Other
- nebo.mobi Other
- uakinogo.io Other
- nv.gov Government
- theweek.com Other
- nofeu.com Other
- 0lin.com Other
- azureserv.com Other
- overheid.nl Other
- niconsul.com Other
- law.com Other
- pinayflix.uk Other
- ilgazzettino.it Other
- sangtacviet.app Other
- mgln.ai Other
- fonts.net Other
- betproexch.live Finance
- alastonsuomi.com Other
- glip.com Other
- oxfordmail.co.uk Other
- twiket.com Other
- tencentcs.com Other
- fireeye.com Other
- z.com Other
- windowscentral.com Other
- sme.sk Other
- ovid.com Other
- ocregister.com Other
- parimatchs123.com Other
- clicktripz.com Other
- omappapi.com Other
- radar.io Other
- inverse.com Other
- intechopen.com Other
- etahub.com Other
- free-dating4you.com Other
- posteo.de News & media
- ebird.org Other
- livestrong.com Other
- newspapers.com News & media
- archive-it.org Other
- babycenter.com Other
- juniper.net Other
- trycloudflare.com Other
- taxtelecom.ru Other
- arcor.de Other
- orlandosentinel.com Other
- educationboardresults.gov.bd Government
- airbrake.io Other
- timesnownews.com News & media
- getrockerbox.com Other
- freesound.org Other
- sun-sentinel.com Other
- z-library.sk Other
- manuaisescolares.pt Other
Showing the 200 most popular of 267. The full list is in sites.csv.
Sites that block Applebot on some paths (488)
- facebook.com Social
- instagram.com Social
- mail.ru Tech
- twitter.com Social
- dzen.ru Other
- linkedin.com Social
- netflix.com Entertainment
- x.com Social
- pinterest.com Social
- adobe.com Tech
- yandex.ru Tech
- samsung.com Tech
- snapchat.com Social
- vk.com Social
- tumblr.com Social
- nytimes.com News & media
- vk.ru Social
- flickr.com Other
- ebay.com E-commerce
- android.com Tech
- imdb.com Reference
- booking.com Travel
- reuters.com News & media
- alibaba.com E-commerce
- drom.ru Other
- trustpilot.com Other
- wsj.com News & media
- dailymotion.com Entertainment
- arxiv.org Reference
- cnbc.com News & media
- sohu.com Tech
- quora.com Reference
- gosuslugi.ru Other
- capcut.com Other
- trendmicro.com Tech
- time.com News & media
- rakuten.co.jp E-commerce
- tripadvisor.com Travel
- threads.com Social
- avito.ru E-commerce
- linode.com Tech
- shopee.com.br E-commerce
- perplexity.ai AI
- fb.com Other
- ya.ru Tech
- airbnb.com Travel
- bestbuy.com E-commerce
- ameblo.jp Social
- disneyplus.com Entertainment
- deepl.com AI
- agoda.com Travel
- academia.edu Reference
- att.com Telecom
- binance.com Finance
- note.com Social
- change.org Social
- theatlantic.com News & media
- ouest-france.fr News & media
- ameba.jp Social
- archiveofourown.org Social
Showing the 60 most popular of 488. The full list is in sites.csv.
Trend
The daily series for Applebot starts on 2026-11-16; a trend appears here as days accumulate.
Latest changes
- pewresearch.org: partial → allowed
- gutenberg.org: allowed → blocked
- wordreference.com: partial → blocked
- athome.co.jp: partial → blocked
- yclients.com: allowed → blocked
- dreamwidth.org: allowed → blocked
- passportindia.gov.in: blocked → allowed
Block or allow Applebot
User-agent: Applebot Disallow: /
Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.
FAQ
- How many websites block Applebot?
- As of 2026-11-16, 267 of the 8,313 popular websites whose robots.txt we could read (3.2%) block Applebot, Apple's AI search crawler, on the home page; 488 more block it on some paths. Among the top 100 sites: 1.2%.
- What is Applebot?
- Applebot is Apple's AI search crawler: it indexes pages so an AI search product can find and cite them. It is the #27 most blocked of the 29 AI crawlers in the index.
- How do I block Applebot in robots.txt?
- Add these lines to https://<your-site>/robots.txt: "User-agent: Applebot" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
- How do I check whether my site blocks Applebot?
- Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of Applebot and 28 other AI crawlers with the exact robots.txt rule.
Other AI crawlers
Also from Apple: Applebot-Extended (13.4%).
- CCBot 16.4%
- Bytespider 16.2%
- GPTBot 15.4%
- ClaudeBot 14.7%
- Meta-ExternalAgent 13.6%
- Applebot-Extended 13.4%
- Google-Extended 13.2%
- Amazonbot 13.2%
- cohere-ai 12.7%
- Google-CloudVertexBot 8.8%
- PerplexityBot 8.2%
- ChatGPT-User 7.4%
- MistralAI-Training 7.1%
- meta-externalfetcher 6.5%
- Perplexity-User 6.1%
- Claude-User 5.9%
- DuckAssistBot 5.9%
- OAI-SearchBot 5.8%
- Claude-SearchBot 5.7%
- MistralAI-User 5.7%
- meta-webindexer 5.1%
- Amzn-SearchBot 4.4%
- Amzn-User 4.4%
- Google-Agent 3.6%
- MistralAI-Index 3.6%
- Google-GeminiNotebook 3.3%
- OAI-AdsBot 3%
- meta-externalads 2.9%