TidyToolsAI Crawler Index · data as of · updated daily

AI crawler access · wfwf436.com

Does wfwf436.com block GPTBot and other AI crawlers?

We could not read the robots.txt of wfwf436.com from our datacenter IP on 2026-11-16 (an error page, a bot wall or a connection problem), so its AI crawler rules are unknown.

All 29 AI crawlers on wfwf436.com

Status for the home page (/) under wfwf436.com's robots.txt. Partially blocked = home page allowed, some paths disallowed.

CrawlerOperatorrobots.txt
Training crawlers
GPTBotOpenAIunknown
ClaudeBotAnthropicunknown
Google-ExtendedGoogleunknown
Applebot-ExtendedAppleunknown
Meta-ExternalAgentMetaunknown
AmazonbotAmazonunknown
MistralAI-TrainingMistralunknown
CCBotCommon Crawlunknown
BytespiderByteDanceunknown
cohere-aiCohereunknown
AI search crawlers
OAI-SearchBotOpenAIunknown
Claude-SearchBotAnthropicunknown
PerplexityBotPerplexityunknown
Google-CloudVertexBotGoogleunknown
ApplebotAppleunknown
meta-webindexerMetaunknown
Amzn-SearchBotAmazonunknown
DuckAssistBotDuckDuckGounknown
MistralAI-IndexMistralunknown
User-triggered fetch crawlers
ChatGPT-UserOpenAIunknown
Claude-UserAnthropicunknown
Perplexity-UserPerplexityunknown
Google-GeminiNotebookGoogleunknown
Google-AgentGoogleunknown
meta-externalfetcherMetaunknown
Amzn-UserAmazonunknown
MistralAI-UserMistralunknown
Ads review crawlers
OAI-AdsBotOpenAIunknown
meta-externaladsMetaunknown

History

6 recorded changes since daily tracking started on 2026-10-02:

DateWhatBeforeAfter
2026-11-16(llms.txt)truefalse
2026-11-15(llms.txt)truefalse
2026-11-14(llms.txt)truefalse
2026-11-13(llms.txt)truefalse
2026-10-07overall policyNo robots.txtOpen
2026-10-07(llms.txt)falsetrue

FAQ

Does wfwf436.com block GPTBot?
Unknown. https://wfwf436.com/robots.txt could not be read on 2026-11-16.
Does wfwf436.com block ClaudeBot?
Unknown. https://wfwf436.com/robots.txt could not be read on 2026-11-16.
Does wfwf436.com block PerplexityBot?
Unknown. https://wfwf436.com/robots.txt could not be read on 2026-11-16.
Can ChatGPT search (OAI-SearchBot) crawl wfwf436.com?
Unknown. https://wfwf436.com/robots.txt could not be read on 2026-11-16.
Does wfwf436.com block Google-Extended (Gemini training)?
Unknown. https://wfwf436.com/robots.txt could not be read on 2026-11-16.
How can I check wfwf436.com or my own site again?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker (no sign-up). To check many sites and get alerts when rules change, use the AI Crawler Access Checker on Apify.

Similar .com sites

All 10,000 sites · All 29 AI crawlers · AI Crawler Index