Answers to common web data questions
Each page answers one question in its first sentence (what you give, what you get, what it costs), then lists the steps, the specifics and the questions people ask next. Numbers come from our tools' documentation and published benchmarks.
Last updated
How to check if GPTBot is blocked on my site
Open https://tools.yukai.uk/ai-crawlers/yourdomain.com (free, no sign-up) or run the AI Crawler Access Checker on Apify for up to 10,000 domains: each site gets GPTBot's status (allowed, partly blocked, blocked or unknown) from its robots.txt, the rule that matched and a ready-to-paste fix, for $2 per 1,000 sites on Apify.
How to convert a website to Markdown for RAG
Give the site's start URL (a docs site, help center or blog) to the Website to Markdown Crawler on Apify and get clean Markdown for every page, RAG chunks with heading paths and one combined OUTPUT.md, for $1.50 per 1,000 pages ($2.50 per 1,000 when a page needs a real browser); failed, blocked and duplicate pages are free.
Bulk audio transcription API: transcribe a list of files
Send a list of audio or video links (MP3, MP4, podcast RSS feeds, YouTube, Google Drive or Dropbox) to the Audio Transcriber on Apify and get each file back as text with timestamps plus SRT, VTT, TXT and Markdown files, for $0.006 per audio minute (Whisper large-v3-turbo, 90+ languages); failed or silent files are free.
How to transcribe a podcast to SRT subtitles
Paste the podcast's RSS feed (or episode file links) into the Audio Transcriber on Apify and each episode comes back as timestamped text with a ready SRT and VTT subtitle file, for $0.006 per audio minute: a 60-minute episode costs $0.36.
How to extract emails from Google Maps leads
Run any Google Maps scraper, then give that run's dataset ID to the Website Email Extractor on Apify: it reads each place's own website and adds the best email, all emails, phone numbers and social profiles to every place, for $2 per 1,000 websites; places with nothing found are free.
How to find contact emails from a list of company websites
Paste the domains into the Website Email Extractor on Apify (one per line, up to 50,000 per run) and get one row per website with the best email, all emails from its home, contact, imprint and about pages, phone numbers and social profiles, for $2 per 1,000 websites; websites with nothing found are free.
How to validate an email list in bulk with an API
Send the addresses (a list, a CSV file or another run's dataset) to the Bulk Email Validator on Apify and get a verdict (valid, risky or invalid), a 0–100 score and a reason for every address from syntax, domain, MX, disposable, role and typo checks, for $0.50 per 1,000 addresses, without SMTP pings.
How to scrape Greenhouse and Lever job postings
Give the Career Site Jobs Actor on Apify a list of company domains, career pages or board URLs (boards.greenhouse.io/<slug>, jobs.lever.co/<slug>) and get every live job posting in one normalised table (title, company, location, salary, posted date, job URL), for $2 per 1,000 jobs; companies without a readable board are free.
YouTube transcript API without an API key
Paste YouTube video, channel or playlist links into the YouTube Transcript Scraper on Apify (no YouTube API key, cookies or login) and get the full transcript, timestamped segments and video details, one row per video, for $3 per 1,000 transcripts; videos without a transcript and failed videos are free.
How to detect a website's tech stack in bulk
Paste a list of domains (or a lead-list dataset) into the Tech Stack Detector on Apify and get each site's CMS, e-commerce platform, frameworks, analytics, hosting and e-mail provider, with the evidence for every detection, for $7 per 1,000 websites; sites with nothing detected and failed sites are free.