What’s the best way to build a reliable node.js website scraper? or Need help optimizing my node.js w

18 Replies, 1439 Views

Rate limits are the worst! I use `proxy-chain` + random delays between requests. Also, respect `robots.txt`—some sites are cool if you play nice.

For speed, try `playwright` instead of Puppeteer. It’s newer and sometimes faster for scraping.

Dynamic content? `page.waitForXPath()` is clutch.

Messages In This Thread



Users browsing this thread: 1 Guest(s)