Dynamic content is the bane of my existence lol. I’ve been using RSelenium for the heavy lifting, but it’s definitely overkill for smaller stuff.
For lighter JS, I’ve had good luck with rvest + V8. Also, polite is a must for ethical scraping—it’s saved me from getting blocked more times than I can count.
If you’re feeling adventurous, check out htmlunit—it’s a nice middle ground between rvest and Selenium.
For lighter JS, I’ve had good luck with rvest + V8. Also, polite is a must for ethical scraping—it’s saved me from getting blocked more times than I can count.
If you’re feeling adventurous, check out htmlunit—it’s a nice middle ground between rvest and Selenium.