Wow, thanks for all the suggestions! Didn’t expect so many options lol.
I tried BeautifulSoup first and it worked for a small test, but hit a block fast. Gonna give Scrapy a shot next since so many of you recommended it.
Quick Q: Anyone used ScrapingBee before? Is it worth paying for if I’m just doing medium-scale scraping?
Also, big thanks for the anti-block tips—didn’t even think about user-agent rotation! 🙏
I tried BeautifulSoup first and it worked for a small test, but hit a block fast. Gonna give Scrapy a shot next since so many of you recommended it.
Quick Q: Anyone used ScrapingBee before? Is it worth paying for if I’m just doing medium-scale scraping?
Also, big thanks for the anti-block tips—didn’t even think about user-agent rotation! 🙏
