Yo! I feel your pain. Using proxy with scrapy is like playing whack-a-mole sometimes. 😂
I’d recommend tweaking your delay settings to at least 3 seconds. Also, try using a proxy pool with at least 10-20 IPs and rotate them frequently.
For tools, check out Scrapy-rotating-proxy or Scrapy-proxy-pool. They’ve saved me a ton of headaches.
Free proxies? Nah, they’re a waste of time unless you’re just testing stuff.
I’d recommend tweaking your delay settings to at least 3 seconds. Also, try using a proxy pool with at least 10-20 IPs and rotate them frequently.
For tools, check out Scrapy-rotating-proxy or Scrapy-proxy-pool. They’ve saved me a ton of headaches.
Free proxies? Nah, they’re a waste of time unless you’re just testing stuff.