Wow, thanks for all the suggestions, everyone! I’ve been testing out Smartproxy and Proxyrack based on your recommendations, and they’re definitely better than what I was using before.
I’m still getting blocked occasionally, though. I’ll try adding some random delays and rotating IPs more frequently like some of you suggested.
Also, has anyone tried using ScrapingBee or Zyte? I’m curious if they’re worth the investment for larger projects.
Thanks again for all the help—this thread has been super useful!
I’m still getting blocked occasionally, though. I’ll try adding some random delays and rotating IPs more frequently like some of you suggested.
Also, has anyone tried using ScrapingBee or Zyte? I’m curious if they’re worth the investment for larger projects.
Thanks again for all the help—this thread has been super useful!