# Siftwright

> Web data APIs for developers and AI agents: YouTube transcripts, website screenshots and PDFs, and Google News results, behind one API key. Only successful results are billed.

- Base URL: https://siftwright.com/v1 (POST JSON, Bearer auth with a key like sw_live_...)
- Endpoints: /v1/youtube-transcript, /v1/screenshot, /v1/google-news, /v1/contact-details, /v1/tech-stack, /v1/app-store-reviews, /v1/ats-jobs, /v1/domain-whois, /v1/pqc-scan (post-quantum TLS readiness, up to 20 hosts per call)
- Plans: Starter $29/mo (10,000 results), Pro $99 (50,000), Business $299 (250,000), Enterprise $999 (1,500,000)
- Pay-per-event alternative on the Apify Store: $1.50 per 1,000 transcripts, captures or articles
- Hosted MCP (Siftwright API key, Streamable HTTP): https://siftwright.com/mcp with header "Authorization: Bearer sw_live_..."; tools youtube_transcript, website_screenshot, google_news, pqc_scan, contact_details, tech_stack, app_store_reviews, ats_jobs, domain_whois
- MCP via Apify: https://mcp.apify.com?tools=siftwright/youtube-transcript-extractor,siftwright/pageframe-screenshots,siftwright/google-news-scraper,siftwright/contact-details-extractor,siftwright/app-store-reviews-scraper,siftwright/tech-stack-detector,siftwright/ats-jobs-scraper,siftwright/domain-whois-dns-lookup,siftwright/email-validator-mx-disposable (Apify token or OAuth)
- OpenAPI 3.1 spec: https://siftwright.com/openapi.json
- Contact: support@siftwright.com

## Docs
- [API docs](https://siftwright.com/docs/): authentication, endpoints, response format, limits, errors
- [OpenAPI spec](https://siftwright.com/openapi.json): machine-readable description of every /v1 endpoint
- [Pricing](https://siftwright.com/pricing/): plans, calculator, pay-per-event comparison

## Products
- [YouTube Transcript API](https://siftwright.com/youtube-transcript-api/)
- [Website Screenshot API](https://siftwright.com/screenshot-api/)
- [Google News API](https://siftwright.com/google-news-api/)
- [HTML to PDF API](https://siftwright.com/html-to-pdf-api/)
- [Web data for AI agents](https://siftwright.com/web-data-for-ai-agents/)
- [Contact Details Extractor](https://siftwright.com/contact-details-extractor/): emails, phones and social links from company websites (Apify Store, $2 per 1,000 websites)
- [Tech Stack Detector](https://siftwright.com/tech-stack-detector/): CMS, frameworks, analytics, hosting, email and DNS provider of any website, with evidence (Apify Store, $2 per 1,000 websites)
- [ATS Jobs Scraper](https://siftwright.com/ats-jobs-scraper/): open jobs from Greenhouse, Lever, Ashby, SmartRecruiters and Recruitee boards in one schema, filtered before charging (Apify Store, $1 per 1,000 jobs)
- [Domain WHOIS & DNS Lookup](https://siftwright.com/domain-whois-lookup/): registrar, creation and expiry dates, age, nameservers, MX, SPF and DMARC for any list of domains via RDAP with WHOIS fallback (Apify Store, $1 per 1,000 domains)
- [App Store Reviews Scraper](https://siftwright.com/app-store-reviews-scraper/): Apple App Store reviews by app name across countries (Apify Store, $0.10 per 1,000 reviews)

## Guides
- [Google News RSS vs a Google News API: what you actually get (with working Python)](https://siftwright.com/blog/google-news-rss-vs-google-news-api/): Google has no official News API, so developers choose between the Google News RSS feed and a third-party API. Here is what each returns in real runs, where the RSS feed falls short, and Python code for both.
- [YouTube subtitles to SRT and VTT from code: language tracks, cue timing and a fallback](https://siftwright.com/blog/youtube-subtitles-srt-vtt-python/): How to get a YouTube video's captions as SRT or VTT files with Python or curl, pick the right language track, fix overlapping cues, and handle the case where translation fails. Output shown is from real runs.
- [Summarise any YouTube video with an LLM: transcript chunking that keeps timestamps](https://siftwright.com/blog/summarize-youtube-video-llm-transcript-chunking/): A tested Python recipe for summarising long YouTube videos with any LLM - fetch timestamped segments, chunk without cutting captions, map-reduce the notes, and turn every cited time into a link that opens the video at that moment.
- [Fix youtube-transcript-api RequestBlocked and IpBlocked on servers](https://siftwright.com/blog/youtube-transcript-api-requestblocked-ipblocked-fix/): Why the Python youtube-transcript-api library works on your laptop but raises RequestBlocked or IpBlocked on AWS, GCP and serverless, which fixes actually work, and a tested fallback that only calls a hosted API when YouTube blocks you.
- [Turn a list of company websites into contact details: emails, phones and socials in Python](https://siftwright.com/blog/contact-details-from-company-websites-python/): Enrich a lead list with public emails, phone numbers and LinkedIn/X links scraped from each company's own site, then rank the emails so you never write to the legal or privacy mailbox. Tested on real sites, including the messy results.
- [Find every Shopify, WordPress and HubSpot site in a lead list: tech stack lookup with evidence in Python](https://siftwright.com/blog/tech-stack-lookup-lead-list-python/): Take a CSV of company websites, detect each site's CMS, ecommerce platform, email provider and hosting, and split the list into segments you can act on. About 70 lines of Python, tested on real sites, with the evidence behind every detection.
- [Find the bugs your users are complaining about: mining App Store reviews with Python](https://siftwright.com/blog/app-store-reviews-python-bug-mining/): Pull 1- and 2-star Apple App Store reviews for any app by name across several countries, group them by app version and tag them with explainable themes, in about 60 lines of Python. Tested on real data.
- [How to get YouTube transcripts with an API (Python, Node.js and curl)](https://siftwright.com/blog/youtube-transcript-api-python-node/): A practical guide to fetching YouTube transcripts from code: why DIY scrapers get IP-blocked on servers, working Python, Node.js and curl examples, languages and translation, SRT/VTT, playlists, and chunking transcripts for LLMs.
- [Feeding clean web data to AI agents with MCP and tool calls](https://siftwright.com/blog/web-data-for-ai-agents-mcp/): How to give AI agents reliable access to YouTube transcripts, web page screenshots and Google News results, through a hosted MCP server or plain tool calling. Setup for Claude, Cursor and VS Code, plus tool design tips that keep context small.
- [Website screenshot API in Python and Node.js: full-page, mobile, retina and PDF](https://siftwright.com/blog/website-screenshot-api-python-node/): How to capture website screenshots from code without running your own headless browser fleet. Working Python and Node.js examples for full-page, mobile, retina and PDF captures, late-loading pages, and a batch script with retries.
- [Build a news monitoring pipeline with a Google News API (Python)](https://siftwright.com/blog/news-monitoring-pipeline-google-news-api/): A step-by-step guide to monitoring brand, competitor and industry news in Python. Design queries, fetch Google News results as JSON, deduplicate, store in SQLite, and send a daily digest, in about 120 lines of code.
- [HTML to PDF at scale: print CSS, fonts, signed URLs and batch rendering](https://siftwright.com/blog/html-to-pdf-at-scale/): A practical guide to generating PDFs from HTML for invoices, reports and certificates. Print CSS that works, page sizes and breaks, waiting for fonts and charts, rendering private documents via signed URLs, and a batch pipeline for thousands of PDFs.
- [Pay-per-result vs subscription API pricing, explained with real numbers](https://siftwright.com/blog/pay-per-result-vs-subscription-api-pricing/): How data API pricing models really compare. Credits, per-request, per-result and subscription pricing explained, the hidden costs to look for, and worked break-even examples using our own two pricing options, including when the cheaper one isn't our subscription.

## Free tools
- [Website tech stack checker](https://siftwright.com/tools/tech-stack-checker/): free check of the CMS, frameworks, analytics, hosting, email and DNS provider of any website, with the evidence for each
- [Website contact finder](https://siftwright.com/tools/contact-finder/): free emails, phone numbers and social links from any company website, CSV/JSON download
- [Website screenshot tool](https://siftwright.com/tools/website-screenshot/): free real-browser screenshot of any public page, desktop/tablet/mobile, full page, PNG or JPEG
- [Google News search & RSS feed builder](https://siftwright.com/tools/google-news/): free Google News search by keyword, time range and country edition, with the matching RSS feed URL and CSV download
- [Email address checker](https://siftwright.com/tools/email-checker/): free check of an email address for syntax, MX records, disposable domains, role addresses and typos (bulk: Bulk Email Validator on the Apify Store, $0.80 per 1,000)
- [Domain WHOIS & DNS lookup](https://siftwright.com/tools/domain-lookup/): free RDAP lookup of registrar, creation/expiry dates, age, nameservers, DNSSEC, MX, SPF and DMARC
- [YouTube transcript generator](https://siftwright.com/tools/youtube-transcript/): free full transcript of one video, TXT/SRT/VTT download
- [robots.txt generator for AI crawlers](https://siftwright.com/tools/robots-txt-generator/): free browser-based generator to allow or block GPTBot, ClaudeBot, PerplexityBot and other AI crawlers, with optional Content-Signal line.
- [AI crawler checker](https://siftwright.com/tools/ai-crawler-checker/): which AI crawlers (GPTBot, ClaudeBot, PerplexityBot...) a site's robots.txt allows, Content-Signal lines and llms.txt validation.
- [Post-quantum TLS checker](https://siftwright.com/tools/pqc-checker/): does a domain support X25519MLKEM768 / ML-KEM key exchange? Also group preference, certificate and HSTS.
- [Qubit Flip](https://siftwright.com/play/qubit-flip/): free daily logic puzzle

## Optional
- [Comparisons](https://siftwright.com/compare/)
- [Changelog](https://siftwright.com/changelog/)
- [Custom data feeds](https://siftwright.com/data/)