Free data
6 open data families, each with one page per website, company or service, rebuilt from public sources every day. No sign-up. For bulk runs, change alerts or your own list of sites, use the paid tools.
AI Crawler Index
Which popular websites block GPTBot, ClaudeBot, PerplexityBot and other AI crawlers in robots.txt, checked every day.
10,000 websites · 29 AI crawlers, 8,295 readable robots.txt files · data of 2026-11-19
Tech Stack Index
What popular websites are built with: CMS, framework, CDN, analytics and more, from public home pages and response headers.
7,649 websites · 393 technologies detected · data of 2026-11-19
Hiring Index
Which companies are hiring now, by role family and country, counted from public job boards of 16 applicant tracking systems.
1,680,438 open jobs · 44,145 companies · data of 2026-11-19
Regional Price Index
What software and subscriptions cost in the US, UK, Singapore and Taiwan, read from each pricing page in that country.
146 services · 4 regions · data of 2026-11-19
Business Lists by City
How many businesses of each kind a city has and how many list a website, from Overture Maps.
2,050,445 businesses · 150 cities, 40 categories · data of 2026-10-03
Funding Tracker
Which companies just raised money, by industry and US state, from the new Form D filings at the SEC, updated daily.
1,506 Form D filings · last 30 days, 31 industries, 50 states · data of 2026-11-19
At a glance
| Dataset | Size | Pages | Data of | Download |
|---|---|---|---|---|
| AI Crawler Index | 10,000 websites | one page per site, per crawler, per category and per country suffix | JSON, CSV | |
| Tech Stack Index | 7,649 websites | one page per site, per technology, per category and comparisons | – | |
| Hiring Index | 1,680,438 open jobs | one page per role family, country, ATS and company | – | |
| Regional Price Index | 146 services | one page per service, category and region pair | – | |
| Business Lists by City | 2,050,445 businesses | one page per city and category | – | |
| Funding Tracker | 1,506 Form D filings | one page per industry and per US state | – |
Data reports: technology usage and business lists →
How the data is made
Every family reads public pages only: robots.txt files, home pages and response headers, the public job boards of applicant tracking systems, and the pricing pages of software products. Each page states the date its numbers were measured. The datasets are published under CC BY 4.0 where a download link is shown; please cite TidyTools.