Struggling with web scraping in R? Any tips or tools to make it easier?

11 Replies, 2441 Views

Wow, thanks so much for all the tips, everyone! I’ve been playing around with RSelenium, and it’s definitely a game-changer for dynamic content. Still getting the hang of it, but it’s way better than banging my head against rvest for JS-heavy sites.

I also tried the `polite` package, and it’s been super helpful for staying respectful of site limits. Still figuring out the whole proxy thing, but I’ll look into some of the services you guys mentioned.

One quick follow-up: anyone have experience with `chromote`? I’m curious how it compares to RSelenium for lighter tasks.

Thanks again, y’all are the best! 🙌

Messages In This Thread



Users browsing this thread: 1 Guest(s)