lol yeah the spider analogy is spot on!
Web crawling is about discovering URLs, while web scraping is about extracting data FROM those URLs. For your project, you’d crawl to find all the product pages, then scrape to get prices/reviews.
If you’re working with multiple sites, you might need proxies (like Luminati) to avoid getting blocked. Also, check if the sites have APIs—way easier than scraping!
Web crawling is about discovering URLs, while web scraping is about extracting data FROM those URLs. For your project, you’d crawl to find all the product pages, then scrape to get prices/reviews.
If you’re working with multiple sites, you might need proxies (like Luminati) to avoid getting blocked. Also, check if the sites have APIs—way easier than scraping!
