Hey! I’ve been using Scrapy for my coupon codescraper, and it’s been working pretty well. The built-in middleware for handling retries and delays is a game-changer.
For proxies, I use free ones from ProxyScrape, but you might wanna invest in paid ones if you’re scraping a lot.
Also, make sure to respect robots.txt—some sites will block you faster if you ignore it.
For proxies, I use free ones from ProxyScrape, but you might wanna invest in paid ones if you’re scraping a lot.
Also, make sure to respect robots.txt—some sites will block you faster if you ignore it.