web-extraction-cascade

Teaches a tiered approach to extracting content from public web pages, trying a readability extractor first, then plain HTTP with an HTML parser, then a headless browser only when JavaScript is required, with checks that halt the run on paywalls, CAPTCHAs, rate limits and block pages. Also covers finding hidden JSON endpoints through browser network tools, polite per-domain pacing, caching, retries, and metadata-first harvesting of public video pages. Use when a task needs text, fields or metadata pulled from public web pages or public video URLs, when plain requests return empty or blocked pages, or when page content only appears after scripts run. Not for logged-in or private data, for defeating paywalls, CAPTCHAs or bot checks, for collecting personal data, or for high-volume crawling.

Version

v1.0.0

Size

16 KB

Updated

2026-10-06

Your client verifies the signature before reading. This explicit manual read is available on every plan. Free automatically loads five chosen skills; paid plans automatically load relevant skills across the library.