Hey! I’ve been using rvest for a while now, and it’s still my go-to for scraping website R. But yeah, dynamic content is a pain.
For that, I’ve started using V8 with rvest. It’s lightweight compared to RSelenium and handles JavaScript-heavy sites pretty well.
Also, for anti-scraping, rotating user agents and proxies is a must. I use proxyR to manage that. It’s not perfect, but it gets the job done.
Hope that helps!
For that, I’ve started using V8 with rvest. It’s lightweight compared to RSelenium and handles JavaScript-heavy sites pretty well.
Also, for anti-scraping, rotating user agents and proxies is a must. I use proxyR to manage that. It’s not perfect, but it gets the job done.
Hope that helps!
