What’s the best way to handle screen scraping in Node.js? or How can I efficiently do screen scraping

Wow, thanks everyone! Didn’t expect so many solid replies.

Tried Playwright based on the recs, and it’s way smoother than Puppeteer for my use case. Also set up some proxies + random delays, and it’s working way better.

Still hitting a few CAPTCHAs though—might check out 2captcha or ScraperAPI like some of you suggested.

Btw, anyone got tips for handling sessions/cookies? Some sites log me out mid-scrape. 😅

(And yeah, checked `robots.txt`—good call!)