Puppeteer is my go-to for screen scraping with Node.js, especially for dynamic sites. Yeah, it’s heavy, but it’s the most reliable for JS-rendered content.
For lighter stuff, Cheerio + axios works great if the site’s static.
To avoid blocks, I use rotating proxies (BrightData’s good) and random delays between requests. Also, spoofing headers helps a ton.
Pro tip: Don’t hammer the site—space out requests or you’ll get banned fast.
For lighter stuff, Cheerio + axios works great if the site’s static.
To avoid blocks, I use rotating proxies (BrightData’s good) and random delays between requests. Also, spoofing headers helps a ton.
Pro tip: Don’t hammer the site—space out requests or you’ll get banned fast.