Should You Do Web Scraping in the Back End? Pros, Cons, and Best Practices or Is It a Good Idea to Do

Backend scraping can work, but it’s like juggling chainsaws.

Pros: Centralized control, no frontend drama.
Cons: Your server becomes a scraping zombie.

Try splitting it—use Celery for task queues or Kubernetes for isolated scraping pods.

Also, check out Zyte (formerly Scrapinghub) if you want a managed solution. Saves time, but costs $$$.