Wow, thanks so much for all the suggestions, everyone! I tried Selenium like a few of you mentioned, and it’s working way better than what I was using before.
I’m still figuring out the whole proxy thing to avoid blocks, but I’ll check out the libraries and tools you guys recommended.
Quick question though—anyone know if Yahoo has a rate limit for scraping? Like, how many requests per minute is safe?
Thanks again, you all rock!
I’m still figuring out the whole proxy thing to avoid blocks, but I’ll check out the libraries and tools you guys recommended.
Quick question though—anyone know if Yahoo has a rate limit for scraping? Like, how many requests per minute is safe?
Thanks again, you all rock!
