TidyToolsAI Crawler Index · data as of · updated daily

AI Crawler Index · domain suffix

Do .tv websites block AI crawlers?

As of 2026-10-03, 6% of the 50 .tv websites whose robots.txt we could read block GPTBot, 6% block ClaudeBot and 4% block PerplexityBot (all 8,357 sites: 12%, 11%, 8.3%). 0 of them (0%) serve an llms.txt (all sites: 15.2%).

6%
block GPTBot (all sites 12%)
6%
block ClaudeBot (all sites 11%)
0%
serve an llms.txt (all sites 15.2%)
50
of 53 sites with a readable robots.txt

Block rate by crawler: .tv vs all sites

CrawlerOperator.tvAll sitesDifference
AmazonbotAmazon8%9.2%-1.2 pp
ApplebotApple6%3.1%+2.9 pp
Applebot-ExtendedApple6%9.6%-3.6 pp
BytespiderByteDance6%12.3%-6.3 pp
CCBotCommon Crawl6%12.8%-6.8 pp
ClaudeBotAnthropic6%11%-5 pp
GPTBotOpenAI6%12%-6 pp
Meta-ExternalAgentMeta6%9.5%-3.5 pp
ChatGPT-UserOpenAI4%7.5%-3.5 pp
Claude-UserAnthropic4%5.5%-1.5 pp
cohere-aiCohere4%9.1%-5.1 pp
Google-ExtendedGoogle4%9.5%-5.5 pp
MistralAI-UserMistral4%5.4%-1.4 pp
OAI-SearchBotOpenAI4%5.7%-1.7 pp
PerplexityBotPerplexity4%8.3%-4.3 pp
Amzn-SearchBotAmazon2%4.1%-2.1 pp
Amzn-UserAmazon2%3.9%-1.9 pp
Claude-SearchBotAnthropic2%5.4%-3.4 pp
DuckAssistBotDuckDuckGo2%5.8%-3.8 pp
Google-AgentGoogle2%3.2%-1.2 pp
Google-CloudVertexBotGoogle2%5.2%-3.2 pp
Google-GeminiNotebookGoogle2%3%-1 pp
meta-externaladsMeta2%2.8%-0.8 pp
meta-externalfetcherMeta2%6%-4 pp
meta-webindexerMeta2%4.6%-2.6 pp
MistralAI-IndexMistral2%3.3%-1.3 pp
MistralAI-TrainingMistral2%3.3%-1.3 pp
OAI-AdsBotOpenAI2%3%-1 pp
Perplexity-UserPerplexity2%5.7%-3.7 pp

Who blocks what, by name: GPTBot, ClaudeBot, CCBot, Bytespider, Applebot-Extended.

Policy mix

PolicySitesShare
Open3566%
No robots.txt1018.9%
Unreadable35.7%
Blocks some AI search23.8%
Blocks training only23.8%
Blocks all AI search11.9%

Most restrictive .tv sites

  1. arte.tv 29 of 29 blocked
  2. france.tv 15 of 29 blocked
  3. avple.tv 6 of 29 blocked
  4. nimo.tv 2 of 29 blocked
  5. best-hit.tv 1 of 29 blocked

All 53 .tv sites

✕ blocked · ◐ partially blocked (some paths) · ✓ allowed · ? robots.txt unreadable. "#" is the position in our popularity-ordered sample.

#SitePolicyGPTBotOAI-SearchChatGPT-UserClaudeBotClaude-SearchPerplexityGoogle-Ext.CCBotllms.txt
97twitch.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
779plex.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
782kingbokep.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
835yasyadong02.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
840kolkataff.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
887pluto.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
1202zeasn.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
1218samba.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
1580medal.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
1647bigolive.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
1777ispot.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
2276okko.tvUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
2670ustream.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
2796arte.tvBlocks all AI searchblockedblockedblockedblockedblockedblockedblockedblocked–
2816pscp.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
3316fubo.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
3519devline.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
3591nimo.tvBlocks some AI searchallowedallowedallowedallowedallowedallowedallowedallowed–
3874abema.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
4131ome.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4169twitcasting.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4391machotube.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4580hanime.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4766vhx.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4825camwhores.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4843boyfriend.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
4933france.tvBlocks some AI searchblockedblockedblockedblockedallowedblockedblockedblocked–
5216redwap.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5232delishkitchen.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5268camstreams.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5517gemidos.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
5526beastiality.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5599wco.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
5642avple.tvBlocks training onlyblockedallowedallowedblockedallowedallowedallowedblocked–
5676wav.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5683superawesome.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5733mykadri.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5784best-hit.tvBlocks training onlyallowedallowedallowedallowedallowedallowedallowedallowed–
5853qfilm.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
5941javmix.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
6109naijaprey.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
6114waipu.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
6276metabet.tvUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
6417huan.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
6560flingo.tvUnreadableunknownunknownunknownunknownunknownunknownunknownunknown–
6846hornyleak.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
7285brightline.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
7784clipro.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
8039ottera.tvNo robots.txtallowedallowedallowedallowedallowedallowedallowedallowed–
8201iwara.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
9136auctor.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
9358peers.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–
9878ren.tvOpenallowedallowedallowedallowedallowedallowedallowedallowed–

Categories on .tv

FAQ

Do .tv websites block AI crawlers?
As of 2026-10-03, 6% of the 50 .tv websites whose robots.txt we could read block GPTBot, 6% block ClaudeBot and 4% block PerplexityBot (all 8,357 sites: 12%, 11%, 8.3%). 0 of them (0%) serve an llms.txt (all sites: 15.2%).
Which .tv websites block the most AI crawlers?
arte.tv (29 of 29), france.tv (15 of 29), avple.tv (6 of 29), nimo.tv (2 of 29), best-hit.tv (1 of 29). The full per-site status is on each site page.
How is this measured?
We fetch each site's robots.txt every day from one residential IP in Taiwan with an honest user agent, and apply the rules to 29 AI crawlers for the home page. 53 .tv sites are in the sample, 50 had a readable robots.txt (3 did not answer or returned an error page).

Other countries and suffixes

Compare all suffixes