What’s the best way to handle node js web scraping without getting blocked? or How can I optimize my

18 Replies, 1632 Views

Honestly, half the battle is respecting robots.txt. Some sites will block you no matter what, so check that first.

For node js web scraping, I combo cheerio with random delays and proxy rotation (I like Storm Proxies).

And hey, if a site’s really aggressive, maybe just… don’t scrape it? Lol.

Messages In This Thread



Users browsing this thread: 1 Guest(s)