Yo, thanks everyone for the awesome tips!
Tried Cheerio + axios first and it worked like a charm for the static sites. Puppeteer was a bit intimidating, but I got it to work after banging my head against the docs for a bit.
One follow-up: How do you guys handle pagination? Like, if I wanna scrape multiple pages, is there a clean way to loop through ’em without getting blocked?
Also, big shoutout for the Playwright and Apify suggestions—gonna check those out next!
Tried Cheerio + axios first and it worked like a charm for the static sites. Puppeteer was a bit intimidating, but I got it to work after banging my head against the docs for a bit.
One follow-up: How do you guys handle pagination? Like, if I wanna scrape multiple pages, is there a clean way to loop through ’em without getting blocked?
Also, big shoutout for the Playwright and Apify suggestions—gonna check those out next!