TidyToolsAI Crawler Index · data as of · updated daily

AI crawler access · kniga-online.com

Does kniga-online.com block GPTBot and other AI crawlers?

We could not read the robots.txt of kniga-online.com from our datacenter IP on 2026-11-16 (an error page, a bot wall or a connection problem), so its AI crawler rules are unknown.

All 29 AI crawlers on kniga-online.com

Status for the home page (/) under kniga-online.com's robots.txt. Partially blocked = home page allowed, some paths disallowed.

CrawlerOperatorrobots.txt
Training crawlers
GPTBotOpenAIunknown
ClaudeBotAnthropicunknown
Google-ExtendedGoogleunknown
Applebot-ExtendedAppleunknown
Meta-ExternalAgentMetaunknown
AmazonbotAmazonunknown
MistralAI-TrainingMistralunknown
CCBotCommon Crawlunknown
BytespiderByteDanceunknown
cohere-aiCohereunknown
AI search crawlers
OAI-SearchBotOpenAIunknown
Claude-SearchBotAnthropicunknown
PerplexityBotPerplexityunknown
Google-CloudVertexBotGoogleunknown
ApplebotAppleunknown
meta-webindexerMetaunknown
Amzn-SearchBotAmazonunknown
DuckAssistBotDuckDuckGounknown
MistralAI-IndexMistralunknown
User-triggered fetch crawlers
ChatGPT-UserOpenAIunknown
Claude-UserAnthropicunknown
Perplexity-UserPerplexityunknown
Google-GeminiNotebookGoogleunknown
Google-AgentGoogleunknown
meta-externalfetcherMetaunknown
Amzn-UserAmazonunknown
MistralAI-UserMistralunknown
Ads review crawlers
OAI-AdsBotOpenAIunknown
meta-externaladsMetaunknown

History

19 recorded changes since daily tracking started on 2026-10-02:

DateWhatBeforeAfter
2026-11-16(content-signal)search=yes,ai-train=no,use=reference
2026-11-15(content-signal)search=yes,ai-train=no,use=reference
2026-11-14(content-signal)search=yes,ai-train=no,use=reference
2026-11-13(content-signal)search=yes,ai-train=no,use=reference
2026-11-12(content-signal)search=yes,ai-train=no,use=reference
2026-11-11(content-signal)search=yes,ai-train=no,use=reference
2026-11-10(content-signal)search=yes,ai-train=no,use=reference
2026-11-09(content-signal)search=yes,ai-train=no,use=reference
2026-11-08(content-signal)search=yes,ai-train=no,use=reference
2026-11-07(content-signal)search=yes,ai-train=no,use=reference
2026-11-06(content-signal)search=yes,ai-train=no,use=reference
2026-11-05(content-signal)search=yes,ai-train=no,use=reference
2026-11-04(content-signal)search=yes,ai-train=no,use=reference
2026-11-03(content-signal)search=yes,ai-train=no,use=reference
2026-11-02(content-signal)search=yes,ai-train=no,use=reference
2026-11-01(content-signal)search=yes,ai-train=no,use=reference
2026-10-31(content-signal)search=yes,ai-train=no,use=reference
2026-10-30(content-signal)search=yes,ai-train=no,use=reference
2026-10-28(content-signal)search=yes,ai-train=no,use=reference

FAQ

Does kniga-online.com block GPTBot?
Unknown. https://kniga-online.com/robots.txt could not be read on 2026-11-16.
Does kniga-online.com block ClaudeBot?
Unknown. https://kniga-online.com/robots.txt could not be read on 2026-11-16.
Does kniga-online.com block PerplexityBot?
Unknown. https://kniga-online.com/robots.txt could not be read on 2026-11-16.
Can ChatGPT search (OAI-SearchBot) crawl kniga-online.com?
Unknown. https://kniga-online.com/robots.txt could not be read on 2026-11-16.
Does kniga-online.com block Google-Extended (Gemini training)?
Unknown. https://kniga-online.com/robots.txt could not be read on 2026-11-16.
How can I check kniga-online.com or my own site again?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker (no sign-up). To check many sites and get alerts when rules change, use the AI Crawler Access Checker on Apify.

Similar .com sites

What is kniga-online.com built with? (Tech Stack Index)

All 10,000 sites · All 29 AI crawlers · AI Crawler Index