Wow, thanks so much for all the replies, guys! I’ve been trying out Bright Data and Scrapy based on your suggestions, and it’s working way better than my old setup. Still getting the hang of rotating IPs, but it’s def helping with the blocks.
Quick question though: how often do you guys rotate your IPs? Like, is there a sweet spot, or does it depend on how much you’re scraping?
Also, I’ll check out ZenRows and Soax next. Thanks again for all the help! 🙌
Quick question though: how often do you guys rotate your IPs? Like, is there a sweet spot, or does it depend on how much you’re scraping?
Also, I’ll check out ZenRows and Soax next. Thanks again for all the help! 🙌