Using proxy with scrapy is like trying to sneak past a bouncer at a club. 😂
I’d say rotate proxies every 5 requests and set a delay of at least 2 seconds. Also, use a middleware like scrapy-fake-useragent to randomize your user-agent.
For paid proxies, I’ve had good results with Luminati (now Bright Data).
I’d say rotate proxies every 5 requests and set a delay of at least 2 seconds. Also, use a middleware like scrapy-fake-useragent to randomize your user-agent.
For paid proxies, I’ve had good results with Luminati (now Bright Data).