Dynamic content is a pain, no doubt. For web scraping in R, RSelenium is your best bet for JS-heavy pages. It’s slow, but it works.
Another tool to check out is `phantomJS`—it’s a headless browser that can render JS, and there’s an R package for it.
For blocks, I’ve found that using a VPN and rotating IPs helps a lot. Also, make sure you’re respecting robots.txt and not hammering the site too hard.
Good luck!
Another tool to check out is `phantomJS`—it’s a headless browser that can render JS, and there’s an R package for it.
For blocks, I’ve found that using a VPN and rotating IPs helps a lot. Also, make sure you’re respecting robots.txt and not hammering the site too hard.
Good luck!