Free data
6 open data families, each with one page per website, company or service, rebuilt from public sources every day. No sign-up. For bulk runs, change alerts or your own list of sites, use the paid tools.
AI Crawler Index
Which popular websites block GPTBot, ClaudeBot, PerplexityBot and other AI crawlers in robots.txt, checked every day.
10,000 websites · 29 AI crawlers, 8,346 readable robots.txt files · data of 2026-10-04
Tech Stack Index
What popular websites are built with: CMS, framework, CDN, analytics and more, from public home pages and response headers.
7,896 websites · 398 technologies detected · data of 2026-10-04
Hiring Index
Which companies are hiring now, by role family and country, counted from public job boards of 16 applicant tracking systems.
1,540,740 open jobs · 43,432 companies · data of 2026-10-04
Regional Price Index
What software and subscriptions cost in the US, UK, Singapore and Taiwan, read from each pricing page in that country.
152 services · 4 regions · data of 2026-10-05
Business Lists by City
How many businesses of each kind a city has and how many list a website, from Overture Maps.
2,050,445 businesses · 150 cities, 40 categories · data of 2026-10-03
Funding Tracker
Which companies just raised money, by industry and US state, from the new Form D filings at the SEC, updated daily.
1,343 Form D filings · last 30 days, 29 industries, 51 states · data of 2026-10-05
At a glance
| Dataset | Size | Pages | Data of | Download |
|---|---|---|---|---|
| AI Crawler Index | 10,000 websites | one page per site, per crawler, per category and per country suffix | JSON, CSV | |
| Tech Stack Index | 7,896 websites | one page per site, per technology, per category and comparisons | – | |
| Hiring Index | 1,540,740 open jobs | one page per role family, country, ATS and company | – | |
| Regional Price Index | 152 services | one page per service, category and region pair | – | |
| Business Lists by City | 2,050,445 businesses | one page per city and category | – | |
| Funding Tracker | 1,343 Form D filings | one page per industry and per US state | – |
How the data is made
Every family reads public pages only: robots.txt files, home pages and response headers, the public job boards of applicant tracking systems, and the pricing pages of software products. Each page states the date its numbers were measured. The datasets are published under CC BY 4.0 where a download link is shown; please cite TidyTools.