Hey everyone!
So, I’ve been diving into web scraping in R lately, and man, dynamic content is a pain. Like, how do you even deal with pages that load stuff with JavaScript? I’ve tried rvest and httr, but sometimes it feels like I’m just hitting a wall.
Also, anyone else get blocked super quick? I’m trying to be polite with delays and headers, but some sites just hate me lol.
What are your go-to tricks for web scraping in R? Like, do you use RSelenium for dynamic stuff? Or is there a better way to avoid getting blocked?
Would love to hear your thoughts!
Cheers!
So, I’ve been diving into web scraping in R lately, and man, dynamic content is a pain. Like, how do you even deal with pages that load stuff with JavaScript? I’ve tried rvest and httr, but sometimes it feels like I’m just hitting a wall.
Also, anyone else get blocked super quick? I’m trying to be polite with delays and headers, but some sites just hate me lol.
What are your go-to tricks for web scraping in R? Like, do you use RSelenium for dynamic stuff? Or is there a better way to avoid getting blocked?
Would love to hear your thoughts!
Cheers!