Hey! Just wanted to say thanks for all the tips—definitely gonna try BeautifulSoup first since it seems simpler.
I messed around with `time.sleep()` and it helped a ton with blocking. Still getting some errors though, maybe need better headers.
Anyone got a working example of a python program to scrape website data with BS4? Like a GitHub gist or something?
Also, is it worth learning Scrapy later if I need to scale up?
Thanks again, y’all are legends!
I messed around with `time.sleep()` and it helped a ton with blocking. Still getting some errors though, maybe need better headers.
Anyone got a working example of a python program to scrape website data with BS4? Like a GitHub gist or something?
Also, is it worth learning Scrapy later if I need to scale up?
Thanks again, y’all are legends!
