What’s the best way to build a reliable node.js website scraper? or Need help optimizing my node.js w

18 Replies, 1697 Views

If you’re scraping heavy sites, split your node.js website scraper into smaller chunks. Too many requests at once = insta-ban.

Use `user-agents` npm package to randomize headers.

For dynamic stuff, `page.waitForResponse()` is gold if you know the API calls.

Messages In This Thread



Users browsing this thread: 1 Guest(s)