lol i was confused about web crawling vs scraping too until i had to do it for work.
crawling = finding pages (like googlebot)
scraping = taking data from those pages
if u know the exact pages u need, just scrape. no need to crawl. but if u dont, u gotta crawl first.
tools? scrapy is OP for both. for easy stuff, try octoparse or parsehub.
watch out for legal stuff tho. some sites block scrapers. use proxies if u gotta.
crawling = finding pages (like googlebot)
scraping = taking data from those pages
if u know the exact pages u need, just scrape. no need to crawl. but if u dont, u gotta crawl first.
tools? scrapy is OP for both. for easy stuff, try octoparse or parsehub.
watch out for legal stuff tho. some sites block scrapers. use proxies if u gotta.
