Honestly, scraping website R can be a headache, especially with dynamic content. I’ve been using rvest + V8 combo, and it’s been working great for most cases.
For anti-scraping, I’d recommend using Tor with R. It’s a bit slow, but it helps avoid IP bans. Also, make sure to randomize your request headers.
If you’re dealing with super aggressive sites, maybe look into Scrapy (Python) and call it from R using reticulate. It’s a bit of a workaround, but it works.
For anti-scraping, I’d recommend using Tor with R. It’s a bit slow, but it helps avoid IP bans. Also, make sure to randomize your request headers.
If you’re dealing with super aggressive sites, maybe look into Scrapy (Python) and call it from R using reticulate. It’s a bit of a workaround, but it works.