Struggling with web scraping in R? Any tips or tools to make it easier?

11 Replies, 2408 Views

Hey there! For web scraping in R, I’d recommend looking into `httr2` if you haven’t already. It’s an updated version of httr and works better with modern APIs.

For dynamic content, RSelenium is the way to go, but it’s a bit heavy. If you want something lighter, try `htmlunit` via the `rJava` package.

And yeah, getting blocked sucks. Use a VPN or proxy service, and always check the site’s `robots.txt` file to stay on their good side.

Messages In This Thread



Users browsing this thread: 1 Guest(s)