I think should you do web scraping in the back end depends on scale. If it's small stuff, sure, why not? But for heavy lifting, you're asking for trouble.
Ever tried Scrapy? It’s a beast for backend scraping but needs tuning. Or just offload it to a cloud function (AWS Lambda, GCP) so your server doesn’t cry.
Also, rotating proxies (like Luminati) help with blocks. But yeah, if you’re hitting rate limits daily, maybe rethink the whole approach.
Ever tried Scrapy? It’s a beast for backend scraping but needs tuning. Or just offload it to a cloud function (AWS Lambda, GCP) so your server doesn’t cry.
Also, rotating proxies (like Luminati) help with blocks. But yeah, if you’re hitting rate limits daily, maybe rethink the whole approach.