AI crawler · Amazon · Training
Which websites block Amazonbot?
As of 2026-10-01, 112 of the 853 popular websites whose robots.txt we could read (13.1%) block Amazonbot, Amazon's AI training crawler, on the home page; 23 more block it on some paths. Among the top 100 sites: 18.1%.
13.1%
of 853 readable sites block Amazonbot
18.1%
of the top 100
#9
most blocked of 29 AI crawlers
718
sites allow it on the home page
Blocked by category
| Category | Sites read | Block Amazonbot |
|---|---|---|
| News & media | 126 | 42.9% |
| Social | 48 | 22.9% |
| Reference | 43 | 14% |
| Other | 260 | 10.4% |
| E-commerce | 58 | 10.3% |
| Government | 33 | 9.1% |
| Travel | 13 | 7.7% |
| Entertainment | 68 | 4.4% |
| Tech | 154 | 0.6% |
| AI | 9 | 0% |
| Education | 20 | 0% |
| Finance | 16 | 0% |
| Telecom | 5 | 0% |
Popular sites that block Amazonbot (112)
- instagram.com Social
- twitter.com Social
- linkedin.com Social
- netflix.com Entertainment
- x.com Social
- pinterest.com Social
- whatsapp.com Social
- reddit.com Social
- snapchat.com Social
- flickr.com Other
- medium.com News & media
- theguardian.com News & media
- ebay.com E-commerce
- forbes.com News & media
- bbc.com News & media
- meraki.com Tech
- launchpad.net Other
- reuters.com News & media
- nature.com Reference
- weibo.com Social
- weather.com News & media
- trustpilot.com Other
- bloomberg.com News & media
- wsj.com News & media
- cnbc.com News & media
- wired.com News & media
- uol.com.br News & media
- usatoday.com News & media
- tripadvisor.com Travel
- threads.com Social
- telegraph.co.uk News & media
- unesco.org Government
- sagepub.com Reference
- elpais.com News & media
- dailymail.com News & media
- lemonde.fr News & media
- fb.com Other
- cnet.com News & media
- bandcamp.com E-commerce
- bild.de News & media
- zillow.com E-commerce
- apnews.com News & media
- ted.com Entertainment
- welt.de News & media
- people.com News & media
- academia.edu Reference
- nbcnews.com News & media
- onet.pl News & media
- corriere.it News & media
- investopedia.com News & media
- healthline.com Reference
- theatlantic.com News & media
- skroutz.gr E-commerce
- as.com News & media
- bfmtv.com News & media
- huffpost.com News & media
- aftonbladet.se News & media
- gsmarena.com News & media
- nypost.com News & media
- theconversation.com News & media
- ad.nl News & media
- nu.nl News & media
- elconfidencial.com News & media
- vg.no News & media
- spiegel.de News & media
- dcinside.com Social
- pandora.com Entertainment
- gazzetta.it News & media
- hln.be News & media
- emag.ro E-commerce
- themoviedb.org Reference
- ladepeche.fr News & media
- mydramalist.com Reference
- actu.fr News & media
- huffingtonpost.com News & media
- dagbladet.no News & media
- ilmeteo.it News & media
- sapo.pt News & media
- dw.com News & media
- iltalehti.fi News & media
- ekstrabladet.dk News & media
- notion.com Other
- tjk.org Other
- buzzfeed.com Other
- aka.ms Other
- sportfilm800.com News & media
- twkan.com Other
- rocketmoney.dev Other
- qcloud.com Other
- newsweek.com News & media
- newyorker.com Other
- va.gov Government
- blackhub.team Other
- arstechnica.com News & media
- vinted.fr Other
- zdnet.com News & media
- adrta.com Other
- marketwatch.com E-commerce
- merkur.de Other
- thelancet.com Other
- wikihow.com Other
- oreilly.com Other
- fastcompany.com Other
- liquipedia.net Other
- lnkd.in Other
- meb.gov.tr Government
- sogou.com Other
- ign.com Other
- salesforceliveagent.com Other
- timeanddate.com Other
- sedoparking.com Other
- washingtonpost.com News & media
Sites that block Amazonbot on some paths (23)
- facebook.com Social
- nytimes.com News & media
- soundcloud.com Entertainment
- oracle.com Tech
- imdb.com Reference
- tradingview.com News & media
- indeed.com Other
- sohu.com Tech
- statista.com News & media
- quora.com Reference
- trendmicro.com Tech
- squarespace.com Tech
- bestbuy.com E-commerce
- disneyplus.com Entertainment
- jimdo.com Tech
- finn.no E-commerce
- fmkorea.com Social
- amap.com Travel
- deezer.com Other
- meteociel.fr News & media
- betpawa.ug Finance
- kargo.com Other
- hulu.com Entertainment
Trend
The daily series for Amazonbot starts on 2026-10-01; a trend appears here as days accumulate.
Latest changes
- dailymotion.com: blocked → allowed
Block or allow Amazonbot
User-agent: Amazonbot Disallow: /
Replace Disallow: / with Allow: / to let it in. A site that blocks training crawlers but wants to be cited in AI answers usually keeps the search and user-triggered crawlers allowed.
FAQ
- How many websites block Amazonbot?
- As of 2026-10-01, 112 of the 853 popular websites whose robots.txt we could read (13.1%) block Amazonbot, Amazon's AI training crawler, on the home page; 23 more block it on some paths. Among the top 100 sites: 18.1%.
- What is Amazonbot?
- Amazonbot is Amazon's AI training crawler: it collects pages to train AI models. It is the #9 most blocked of the 29 AI crawlers in the index.
- How do I block Amazonbot in robots.txt?
- Add these lines to https://<your-site>/robots.txt: "User-agent: Amazonbot" followed by "Disallow: /". To allow it, use "Allow: /" instead. Well-behaved crawlers read robots.txt before fetching pages; firewall rules are needed to stop crawlers that ignore it.
- How do I check whether my site blocks Amazonbot?
- Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker: it shows the status of Amazonbot and 28 other AI crawlers with the exact robots.txt rule.
Other AI crawlers
Also from Amazon: Amzn-SearchBot (5.9%), Amzn-User (5.5%).
- CCBot 19.5%
- Bytespider 17.7%
- GPTBot 17%
- ClaudeBot 16.3%
- Meta-ExternalAgent 14.4%
- Applebot-Extended 14.3%
- Google-Extended 14%
- cohere-ai 13.7%
- PerplexityBot 12.7%
- ChatGPT-User 9.6%
- Perplexity-User 8.9%
- meta-externalfetcher 8.7%
- MistralAI-User 8.4%
- Claude-User 8.3%
- Claude-SearchBot 8.1%
- DuckAssistBot 8.1%
- Google-CloudVertexBot 7.2%
- OAI-SearchBot 6.7%
- meta-webindexer 6.3%
- Amzn-SearchBot 5.9%
- Amzn-User 5.5%
- MistralAI-Index 4.7%
- MistralAI-Training 4.7%
- Google-Agent 4.5%
- OAI-AdsBot 4.3%
- Google-GeminiNotebook 4.3%
- meta-externalads 3.8%
- Applebot 2.6%