Should You Do Web Scraping in the Back End? Pros, Cons, and Best Practices or Is It a Good Idea to Do

27 Replies, 2312 Views

Back end scraping is a solid choice, but it’s not without headaches.

Biggest issue? Maintenance. Sites change their HTML all the time, and your scraper breaks. If you’re not ready to babysit it, maybe reconsider.

For tools, I’d say go with Scrapy + Splash for JS-heavy sites. And always, ALWAYS use proxies.

Messages In This Thread



Users browsing this thread: 1 Guest(s)