Yo, web scraping in R is a grind, especially with dynamic content. RSelenium is the way to go for JS stuff—it’s a bit clunky, but it works.
Another tip: try `httr2` if you haven’t already. It’s an updated version of httr and has some cool features for handling headers and cookies.
For blocks, I’ve found that using residential proxies (like from Bright Data) makes a huge difference. They’re pricey, but worth it if you’re scraping a lot.
Good luck, and don’t let the blocks get you down!
Another tip: try `httr2` if you haven’t already. It’s an updated version of httr and has some cool features for handling headers and cookies.
For blocks, I’ve found that using residential proxies (like from Bright Data) makes a huge difference. They’re pricey, but worth it if you’re scraping a lot.
Good luck, and don’t let the blocks get you down!