Wow, thanks everyone! Didn’t expect so many tips on how to setup web scrapper.
Tried BeautifulSoup last night and it worked! Scraped my first page (just titles, but still).
Quick Q: How do I know if a site’s okay with scraping? Just robots.txt? Also, what’s a good delay—2 seconds or more?
P.S. The quotes.toscrape.com site was clutch. 🙌
Tried BeautifulSoup last night and it worked! Scraped my first page (just titles, but still).
Quick Q: How do I know if a site’s okay with scraping? Just robots.txt? Also, what’s a good delay—2 seconds or more?
P.S. The quotes.toscrape.com site was clutch. 🙌
