TidyToolsAI Crawler Index · data as of · updated daily

AI crawler access · allegroimg.com

Does allegroimg.com block GPTBot and other AI crawlers?

We could not read the robots.txt of allegroimg.com from our datacenter IP on 2026-11-16 (an error page, a bot wall or a connection problem), so its AI crawler rules are unknown.

All 29 AI crawlers on allegroimg.com

Status for the home page (/) under allegroimg.com's robots.txt. Partially blocked = home page allowed, some paths disallowed.

CrawlerOperatorrobots.txt
Training crawlers
GPTBotOpenAIblocked
ClaudeBotAnthropicblocked
Google-ExtendedGoogleblocked
Applebot-ExtendedAppleblocked
Meta-ExternalAgentMetablocked
AmazonbotAmazonblocked
MistralAI-TrainingMistralblocked
CCBotCommon Crawlblocked
BytespiderByteDanceblocked
cohere-aiCohereblocked
AI search crawlers
OAI-SearchBotOpenAIblocked
Claude-SearchBotAnthropicblocked
PerplexityBotPerplexityblocked
Google-CloudVertexBotGoogleblocked
ApplebotAppleblocked
meta-webindexerMetablocked
Amzn-SearchBotAmazonblocked
DuckAssistBotDuckDuckGoblocked
MistralAI-IndexMistralblocked
User-triggered fetch crawlers
ChatGPT-UserOpenAIblocked
Claude-UserAnthropicblocked
Perplexity-UserPerplexityblocked
Google-GeminiNotebookGoogleblocked
Google-AgentGoogleblocked
meta-externalfetcherMetablocked
Amzn-UserAmazonblocked
MistralAI-UserMistralblocked
Ads review crawlers
OAI-AdsBotOpenAIblocked
meta-externaladsMetablocked

History

30 recorded changes since daily tracking started on 2026-10-02:

DateWhatBeforeAfter
2026-10-21GPTBotallowedblocked
2026-10-21OAI-SearchBotallowedblocked
2026-10-21ChatGPT-Userallowedblocked
2026-10-21OAI-AdsBotallowedblocked
2026-10-21ClaudeBotallowedblocked
2026-10-21Claude-SearchBotallowedblocked
2026-10-21Claude-Userallowedblocked
2026-10-21PerplexityBotallowedblocked
2026-10-21Perplexity-Userallowedblocked
2026-10-21Google-Extendedallowedblocked
2026-10-21Google-CloudVertexBotallowedblocked
2026-10-21Google-GeminiNotebookallowedblocked
2026-10-21Google-Agentallowedblocked
2026-10-21Applebotallowedblocked
2026-10-21Applebot-Extendedallowedblocked
2026-10-21Meta-ExternalAgentallowedblocked
2026-10-21meta-webindexerallowedblocked
2026-10-21meta-externalfetcherallowedblocked
2026-10-21meta-externaladsallowedblocked
2026-10-21Amazonbotallowedblocked
2026-10-21Amzn-SearchBotallowedblocked
2026-10-21Amzn-Userallowedblocked
2026-10-21DuckAssistBotallowedblocked
2026-10-21MistralAI-Userallowedblocked
2026-10-21MistralAI-Indexallowedblocked
2026-10-21MistralAI-Trainingallowedblocked
2026-10-21CCBotallowedblocked
2026-10-21Bytespiderallowedblocked
2026-10-21cohere-aiallowedblocked
2026-10-21overall policyNo robots.txtBlocks all AI search

FAQ

Does allegroimg.com block GPTBot?
Yes. As of 2026-11-16, https://allegroimg.com/robots.txt disallows the home page for GPTBot, OpenAI's AI training crawler.
Does allegroimg.com block ClaudeBot?
Yes. As of 2026-11-16, https://allegroimg.com/robots.txt disallows the home page for ClaudeBot, Anthropic's AI training crawler.
Does allegroimg.com block PerplexityBot?
Yes. As of 2026-11-16, https://allegroimg.com/robots.txt disallows the home page for PerplexityBot, Perplexity's AI search crawler.
Can ChatGPT search (OAI-SearchBot) crawl allegroimg.com?
Yes. As of 2026-11-16, https://allegroimg.com/robots.txt disallows the home page for OAI-SearchBot, OpenAI's AI search crawler.
Does allegroimg.com block Google-Extended (Gemini training)?
Yes. As of 2026-11-16, https://allegroimg.com/robots.txt disallows the home page for Google-Extended, Google's AI training crawler.
How can I check allegroimg.com or my own site again?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker (no sign-up). To check many sites and get alerts when rules change, use the AI Crawler Access Checker on Apify.

Similar .com sites

All 10,000 sites · All 29 AI crawlers · AI Crawler Index