Thanks so much for all the suggestions, everyone! I’ve been experimenting with Puppeteer and rotating proxies based on your advice, and it’s definitely working better than what I was doing before.
I’m still running into some issues with captchas, though. Has anyone found a reliable way to handle those without getting blocked?
Also, I checked out Octoparse and Apify, and they look promising. I’ll give them a try and see how it goes.
Really appreciate all the help—this thread has been super useful!
I’m still running into some issues with captchas, though. Has anyone found a reliable way to handle those without getting blocked?
Also, I checked out Octoparse and Apify, and they look promising. I’ll give them a try and see how it goes.
Really appreciate all the help—this thread has been super useful!