AI Crawler Index
Which countries' websites block AI crawlers most?
Among country domains with at least 50 readable sites, France (.fr) blocks GPTBot most often (31.8% of 85 sites) and China (.cn) least often (3.8% of 53). Overall 12% of sites block GPTBot. Data of 2026-10-03; suffixes with fewer than 30 readable sites are not listed.
Country domains
| Suffix | Sites read | GPTBot | ClaudeBot | Perplexity | llms.txt |
|---|---|---|---|---|---|
| .fr (France) | 85 | 31.8% | 30.6% | 23.5% | 3.5% |
| .it (Italy) | 55 | 27.3% | 25.5% | 21.8% | 7.3% |
| .nl (Netherlands) | 48 | 27.1% | 25% | 22.9% | 4.2% |
| .au (Australia) | 37 | 27% | 29.7% | 27% | 16.2% |
| .pl (Poland) | 53 | 26.4% | 9.4% | 7.5% | 9.4% |
| .es (Spain) | 36 | 25% | 19.4% | 13.9% | 8.3% |
| .de (Germany) | 203 | 23.6% | 20.2% | 13.3% | 7.9% |
| .uk (United Kingdom) | 90 | 20% | 21.1% | 21.1% | 8.9% |
| .ch (Switzerland) | 36 | 16.7% | 16.7% | 8.3% | 2.8% |
| .ca (Canada) | 42 | 7.1% | 2.4% | 7.1% | 9.5% |
| .br (Brazil) | 90 | 6.7% | 5.6% | 4.4% | 15.6% |
| .jp (Japan) | 149 | 6% | 5.4% | 4.7% | 6% |
| .ru (Russia) | 297 | 5.1% | 2.4% | 1.7% | 10.4% |
| .in (India) | 74 | 4.1% | 1.4% | 1.4% | 10.8% |
| .cn (China) | 53 | 3.8% | 1.9% | 1.9% | 1.9% |
| .cz (Czechia) | 31 | 3.2% | 6.5% | 3.2% | 6.5% |
| .ir (Iran) | 35 | 0% | 0% | 0% | 11.4% |
Generic domains
A country-code domain says where a site is registered, not necessarily where its audience or company is. Sorted by the GPTBot block rate.