TidyToolsAI Crawler Index · data as of · updated daily

AI crawler access · tantora.jp

Does tantora.jp block GPTBot and other AI crawlers?

GPTBot: allowedtantora.jp has no robots.txt file, so all 29 AI crawlers we track, including GPTBot, ClaudeBot and PerplexityBot, are allowed by default (as of 2026-11-16).

All 29 AI crawlers on tantora.jp

Status for the home page (/) under tantora.jp's robots.txt. Partially blocked = home page allowed, some paths disallowed.

CrawlerOperatorrobots.txt
Training crawlers
GPTBotOpenAIallowed
ClaudeBotAnthropicallowed
Google-ExtendedGoogleallowed
Applebot-ExtendedAppleallowed
Meta-ExternalAgentMetaallowed
AmazonbotAmazonallowed
MistralAI-TrainingMistralallowed
CCBotCommon Crawlallowed
BytespiderByteDanceallowed
cohere-aiCohereallowed
AI search crawlers
OAI-SearchBotOpenAIallowed
Claude-SearchBotAnthropicallowed
PerplexityBotPerplexityallowed
Google-CloudVertexBotGoogleallowed
ApplebotAppleallowed
meta-webindexerMetaallowed
Amzn-SearchBotAmazonallowed
DuckAssistBotDuckDuckGoallowed
MistralAI-IndexMistralallowed
User-triggered fetch crawlers
ChatGPT-UserOpenAIallowed
Claude-UserAnthropicallowed
Perplexity-UserPerplexityallowed
Google-GeminiNotebookGoogleallowed
Google-AgentGoogleallowed
meta-externalfetcherMetaallowed
Amzn-UserAmazonallowed
MistralAI-UserMistralallowed
Ads review crawlers
OAI-AdsBotOpenAIallowed
meta-externaladsMetaallowed

Compared with other other sites in the index: 15.1% of them block GPTBot and 14.1% block ClaudeBot (n=6651). Across all 8,313 readable sites: 15.4% block GPTBot. All other sites →

On .jp (Japan) domains: 6.9% of 145 readable sites block GPTBot and 4.8% serve an llms.txt. All .jp (Japan) sites →

History

30 recorded changes since daily tracking started on 2026-10-02:

DateWhatBeforeAfter
2026-10-23GPTBotblockedallowed
2026-10-23OAI-SearchBotblockedallowed
2026-10-23ChatGPT-Userblockedallowed
2026-10-23OAI-AdsBotblockedallowed
2026-10-23ClaudeBotblockedallowed
2026-10-23Claude-SearchBotblockedallowed
2026-10-23Claude-Userblockedallowed
2026-10-23PerplexityBotblockedallowed
2026-10-23Perplexity-Userblockedallowed
2026-10-23Google-Extendedblockedallowed
2026-10-23Google-CloudVertexBotblockedallowed
2026-10-23Google-GeminiNotebookblockedallowed
2026-10-23Google-Agentblockedallowed
2026-10-23Applebotblockedallowed
2026-10-23Applebot-Extendedblockedallowed
2026-10-23Meta-ExternalAgentblockedallowed
2026-10-23meta-webindexerblockedallowed
2026-10-23meta-externalfetcherblockedallowed
2026-10-23meta-externaladsblockedallowed
2026-10-23Amazonbotblockedallowed
2026-10-23Amzn-SearchBotblockedallowed
2026-10-23Amzn-Userblockedallowed
2026-10-23DuckAssistBotblockedallowed
2026-10-23MistralAI-Userblockedallowed
2026-10-23MistralAI-Indexblockedallowed
2026-10-23MistralAI-Trainingblockedallowed
2026-10-23CCBotblockedallowed
2026-10-23Bytespiderblockedallowed
2026-10-23cohere-aiblockedallowed
2026-10-23overall policyBlocks all AI searchNo robots.txt

FAQ

Does tantora.jp block GPTBot?
No. tantora.jp has no robots.txt (checked 2026-11-16), so GPTBot, OpenAI's AI training crawler, is allowed by default.
Does tantora.jp block ClaudeBot?
No. tantora.jp has no robots.txt (checked 2026-11-16), so ClaudeBot, Anthropic's AI training crawler, is allowed by default.
Does tantora.jp block PerplexityBot?
No. tantora.jp has no robots.txt (checked 2026-11-16), so PerplexityBot, Perplexity's AI search crawler, is allowed by default.
Can ChatGPT search (OAI-SearchBot) crawl tantora.jp?
No. tantora.jp has no robots.txt (checked 2026-11-16), so OAI-SearchBot, OpenAI's AI search crawler, is allowed by default.
Does tantora.jp block Google-Extended (Gemini training)?
No. tantora.jp has no robots.txt (checked 2026-11-16), so Google-Extended, Google's AI training crawler, is allowed by default.
Does tantora.jp have an llms.txt file?
No. As of 2026-11-16, tantora.jp does not serve a plain-text /llms.txt file.
How can I check tantora.jp or my own site again?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker (no sign-up). To check many sites and get alerts when rules change, use the AI Crawler Access Checker on Apify.

Similar .jp (Japan) sites

Which .jp (Japan) sites block GPTBot · What is tantora.jp built with? (Tech Stack Index)

All 10,000 sites · All 29 AI crawlers · AI Crawler Index