TidyToolsAI Crawler Index · data as of · updated daily

AI crawler access · igrsup.gov.in

Does igrsup.gov.in block GPTBot and other AI crawlers?

GPTBot: allowedigrsup.gov.in has no robots.txt file, so all 29 AI crawlers we track, including GPTBot, ClaudeBot and PerplexityBot, are allowed by default (as of 2026-10-03).

All 29 AI crawlers on igrsup.gov.in

Status for the home page (/) under igrsup.gov.in's robots.txt. Partially blocked = home page allowed, some paths disallowed.

CrawlerOperatorrobots.txt
Training crawlers
GPTBotOpenAIallowed
ClaudeBotAnthropicallowed
Google-ExtendedGoogleallowed
Applebot-ExtendedAppleallowed
Meta-ExternalAgentMetaallowed
AmazonbotAmazonallowed
MistralAI-TrainingMistralallowed
CCBotCommon Crawlallowed
BytespiderByteDanceallowed
cohere-aiCohereallowed
AI search crawlers
OAI-SearchBotOpenAIallowed
Claude-SearchBotAnthropicallowed
PerplexityBotPerplexityallowed
Google-CloudVertexBotGoogleallowed
ApplebotAppleallowed
meta-webindexerMetaallowed
Amzn-SearchBotAmazonallowed
DuckAssistBotDuckDuckGoallowed
MistralAI-IndexMistralallowed
User-triggered fetch crawlers
ChatGPT-UserOpenAIallowed
Claude-UserAnthropicallowed
Perplexity-UserPerplexityallowed
Google-GeminiNotebookGoogleallowed
Google-AgentGoogleallowed
meta-externalfetcherMetaallowed
Amzn-UserAmazonallowed
MistralAI-UserMistralallowed
Ads review crawlers
OAI-AdsBotOpenAIallowed
meta-externaladsMetaallowed

Compared with other government sites in the index: 3.8% of them block GPTBot and 3.8% block ClaudeBot (n=210). Across all 8,357 readable sites: 12% block GPTBot. All government sites →

On .in (India) domains: 4.1% of 74 readable sites block GPTBot and 10.8% serve an llms.txt. All .in (India) sites →

History

No change recorded since daily tracking started on 2026-10-02. We re-check igrsup.gov.in every day; a change in its AI crawler rules shows up here and on the index page.

FAQ

Does igrsup.gov.in block GPTBot?
No. igrsup.gov.in has no robots.txt (checked 2026-10-03), so GPTBot, OpenAI's AI training crawler, is allowed by default.
Does igrsup.gov.in block ClaudeBot?
No. igrsup.gov.in has no robots.txt (checked 2026-10-03), so ClaudeBot, Anthropic's AI training crawler, is allowed by default.
Does igrsup.gov.in block PerplexityBot?
No. igrsup.gov.in has no robots.txt (checked 2026-10-03), so PerplexityBot, Perplexity's AI search crawler, is allowed by default.
Can ChatGPT search (OAI-SearchBot) crawl igrsup.gov.in?
No. igrsup.gov.in has no robots.txt (checked 2026-10-03), so OAI-SearchBot, OpenAI's AI search crawler, is allowed by default.
Does igrsup.gov.in block Google-Extended (Gemini training)?
No. igrsup.gov.in has no robots.txt (checked 2026-10-03), so Google-Extended, Google's AI training crawler, is allowed by default.
Does igrsup.gov.in have an llms.txt file?
No. As of 2026-10-03, igrsup.gov.in does not serve a plain-text /llms.txt file.
How can I check igrsup.gov.in or my own site again?
Use the free AI crawler checker at tools.yukai.uk/free/ai-crawler-checker (no sign-up). To check many sites and get alerts when rules change, use the AI Crawler Access Checker on Apify.

Similar government sites

Which government sites block GPTBot · Which .in (India) sites block GPTBot

All 10,000 sites · All 29 AI crawlers · AI Crawler Index