TidyToolsAI Crawler Index · data as of · updated daily

AI Crawler Index · country

Do websites in France (.fr) block AI crawlers?

As of 2026-10-05, 32.5% of the 83 websites in France (.fr) whose robots.txt we could read block GPTBot, 31.3% block ClaudeBot and 24.1% block PerplexityBot (all 8,344 sites: 12.1%, 11%, 8.3%). 3 of them (3.6%) serve an llms.txt (all sites: 15.3%).

32.5%
block GPTBot (all sites 12.1%)
31.3%
block ClaudeBot (all sites 11%)
3.6%
serve an llms.txt (all sites 15.3%)
83
of 107 sites with a readable robots.txt

Block rate by crawler: .fr (France) vs all sites

CrawlerOperator.fr (France)All sitesDifference
CCBotCommon Crawl36.1%12.8%+23.3 pp
GPTBotOpenAI32.5%12.1%+20.4 pp
AmazonbotAmazon31.3%9.2%+22.1 pp
ClaudeBotAnthropic31.3%11%+20.3 pp
Google-ExtendedGoogle30.1%9.6%+20.5 pp
BytespiderByteDance27.7%12.3%+15.4 pp
cohere-aiCohere27.7%9.1%+18.6 pp
ChatGPT-UserOpenAI25.3%7.4%+17.9 pp
PerplexityBotPerplexity24.1%8.3%+15.8 pp
Applebot-ExtendedApple21.7%9.6%+12.1 pp
OAI-SearchBotOpenAI20.5%5.7%+14.8 pp
Meta-ExternalAgentMeta19.3%9.5%+9.8 pp
MistralAI-UserMistral19.3%5.4%+13.9 pp
DuckAssistBotDuckDuckGo18.1%5.8%+12.3 pp
Claude-UserAnthropic15.7%5.5%+10.2 pp
meta-externalfetcherMeta15.7%6%+9.7 pp
Claude-SearchBotAnthropic14.5%5.5%+9 pp
Perplexity-UserPerplexity10.8%5.7%+5.1 pp
Google-CloudVertexBotGoogle8.4%5.2%+3.2 pp
meta-webindexerMeta8.4%4.6%+3.8 pp
ApplebotApple6%3.1%+2.9 pp
MistralAI-IndexMistral6%3.4%+2.6 pp
Google-AgentGoogle3.6%3.3%+0.3 pp
Amzn-SearchBotAmazon2.4%4.2%-1.8 pp
Amzn-UserAmazon2.4%3.9%-1.5 pp
Google-GeminiNotebookGoogle2.4%3%-0.6 pp
meta-externaladsMeta2.4%2.9%-0.5 pp
MistralAI-TrainingMistral2.4%3.3%-0.9 pp
OAI-AdsBotOpenAI2.4%3%-0.6 pp

Who blocks what, by name: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, Bytespider, Applebot-Extended, OAI-SearchBot.

Policy mix

PolicySitesShare
Open4542.1%
Blocks some AI search2624.3%
Unreadable2422.4%
Blocks training only76.5%
No robots.txt43.7%
Blocks all AI search10.9%

Most restrictive .fr (France) sites

  1. mailin.fr 29 of 29 blocked
  2. lanouvellerepublique.fr 28 of 29 blocked
  3. franceinfo.fr 19 of 29 blocked
  4. lesechos.fr 19 of 29 blocked
  5. francetvinfo.fr 19 of 29 blocked
  6. actu.fr 17 of 29 blocked
  7. radiofrance.fr 17 of 29 blocked
  8. ici.fr 17 of 29 blocked
  9. lavoixdunord.fr 17 of 29 blocked
  10. francebleu.fr 17 of 29 blocked
  11. lexpress.fr 16 of 29 blocked
  12. lemonde.fr 15 of 29 blocked
  13. tf1info.fr 15 of 29 blocked
  14. 20minutes.fr 14 of 29 blocked
  15. letelegramme.fr 14 of 29 blocked

Sites with an llms.txt (3)

Top 100 of 107 .fr (France) sites

✕ blocked · ◐ partially blocked (some paths) · ✓ allowed · ? robots.txt unreadable. "#" is the position in our popularity-ordered sample.

#SitePolicyGPTBotOAI-SearchChatGPT-UserClaudeBotClaude-SearchPerplexityGoogle-Ext.CCBotllms.txt
255free.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
317lemonde.frBlocks some AI searchallowedallowedallowedblockedblockedallowedblockedblocked–
369lefigaro.frBlocks some AI searchblockedblockedblockedblockedblockedallowedblockedblocked–
484ouest-france.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
556leboncoin.frBlocks training onlypartialpartialpartialpartialpartialpartialpartialpartial–
561lequipe.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
605ladepeche.frBlocks some AI searchblockedallowedallowedblockedallowedblockedblockedblocked–
618actu.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
656leroymerlin.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
760meteociel.frBlocks training onlyallowedallowedallowedblockedpartialallowedallowedallowed–
848vinted.frBlocks training onlyblockedallowedallowedblockedallowedallowedblockedblocked–
978tube8.frNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
1258sfr.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
1380leparisien.frBlocks some AI searchblockedallowedallowedblockedallowedblockedallowedblocked–
1386cnil.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
1461franceinfo.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
1686cnrs.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
1933ec-lyon.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
2100weborama.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
2189rfi.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
2333caissedesdepots.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
2375radiofrance.frBlocks some AI searchblockedblockedblockedblockedallowedblockedblockedblocked–
248820minutes.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
2610liberation.frBlocks some AI searchblockedblockedblockedblockedallowedblockedallowedblocked–
2624bnf.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
2644ameli.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
2735service-public.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
2869laposte.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
2945lesechos.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
3011allocine.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
3088letour.frBlocks some AI searchallowedallowedallowedallowedallowedallowedallowedallowed–
3097sudouest.frBlocks some AI searchblockedallowedblockedallowedallowedallowedallowedblocked–
3135legifrance.gouv.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
3239francetvinfo.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
3245paris.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
3251pagesjaunes.frBlocks training onlyallowedallowedallowedallowedallowedallowedallowedblocked–
3266credit-agricole.frOpenallowedallowedallowedallowedallowedallowedallowedallowedyes
3350inria.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
3494ici.frBlocks some AI searchblockedblockedblockedblockedallowedblockedblockedblocked–
3560tf1info.frBlocks some AI searchblockedallowedblockedblockedallowedallowedblockedblocked–
3572louvre.frOpenpartialallowedpartialpartialallowedallowedpartialpartial–
3581carrefour.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
3585caf.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
3652letelegramme.frBlocks some AI searchblockedblockedblockedblockedallowedblockedblockedblocked–
3656lavoixdunord.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblockedyes
3829decathlon.frOpenallowedallowedallowedpartialallowedallowedallowedallowed–
3837leprogres.frBlocks some AI searchblockedallowedblockedallowedallowedallowedblockedblocked–
3878midilibre.frBlocks some AI searchblockedallowedallowedblockedallowedblockedblockedblocked–
4085cnews.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
4114lanouvellerepublique.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
4265tf1.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
4398auchan.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4469eurosport.frOpenallowedallowedallowedpartialallowedallowedallowedallowed–
4501lindependant.frBlocks some AI searchblockedallowedallowedblockedallowedblockedblockedblocked–
4508creditmutuel.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4596but.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4677labanquepostale.frNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
4704mailin.frBlocks all AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
4711chronopost.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4767fdj.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
4788senat.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
4848conforama.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
4868intersport.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
5060cic.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5152manomano.frOpenallowedallowedallowedallowedallowedallowedallowedallowedyes
5210melty.frOpenallowedallowedallowedallowedallowedallowedallowedpartial–
5224impots.gouv.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5304maxifoot.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
5493sciencepost.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5595naturabuy.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5631zone-turf.frOpenpartialallowedpartialallowedallowedallowedallowedpartial–
5638turfomania.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
5672electrodepot.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5677zeturf.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
6030francetravail.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
6202franceconnect.gouv.frNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
6432lepoint.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
6577insee.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
7042mariefrance.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
7113lexpress.frBlocks some AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
7193unistra.frBlocks training onlypartialallowedpartialpartialallowedallowedallowedallowed–
7393sante.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
7612bbox.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
7788journaldesfemmes.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
7831archives-ouvertes.frBlocks some AI searchblockedblockedblockedblockedallowedblockedblockedblocked–
8122inrae.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
8319laredoute.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
8329ademe.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
8458first-id.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
8500latribune.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
8506assemblee-nationale.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
8539ens.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
8595renater.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
8699inserm.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
8706banquepopulaire.frUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
8770francebleu.frBlocks some AI searchblockedblockedblockedblockedallowedblockedblockedblocked–
8998pappers.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
9183mediapart.frBlocks training onlyblockedallowedallowedblockedallowedallowedblockedblocked–
9337iarc.frOpenallowedallowedallowedallowedallowedallowedallowedallowed–
9511online.frNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–

Sorted by popularity; the rest are on the A-Z lists.

Categories on .fr (France)

FAQ

Do websites in France (.fr) block AI crawlers?
As of 2026-10-05, 32.5% of the 83 websites in France (.fr) whose robots.txt we could read block GPTBot, 31.3% block ClaudeBot and 24.1% block PerplexityBot (all 8,344 sites: 12.1%, 11%, 8.3%). 3 of them (3.6%) serve an llms.txt (all sites: 15.3%).
Which .fr (France) websites block the most AI crawlers?
mailin.fr (29 of 29), lanouvellerepublique.fr (28 of 29), franceinfo.fr (19 of 29), lesechos.fr (19 of 29), francetvinfo.fr (19 of 29). The full per-site status is on each site page.
How is this measured?
We fetch each site's robots.txt every day from one residential IP in Taiwan with an honest user agent, and apply the rules to 29 AI crawlers for the home page. 107 .fr (France) sites are in the sample, 83 had a readable robots.txt (24 did not answer or returned an error page).

Other countries and suffixes

Compare all suffixes