Man, scraping twitter is no joke. I’ve been using Scrapy with a custom middleware to handle proxies and user agents. It’s a bit of work to set up, but once it’s running, it’s pretty smooth.
For proxies, I’ve been using Luminati. It’s not the cheapest, but it’s reliable for scraping twitter data at scale.
Also, if you’re scraping twitter for hashtags or trends, Twint is worth checking out. It’s a bit outdated, but it still works for some use cases.
For proxies, I’ve been using Luminati. It’s not the cheapest, but it’s reliable for scraping twitter data at scale.
Also, if you’re scraping twitter for hashtags or trends, Twint is worth checking out. It’s a bit outdated, but it still works for some use cases.