Best Tools and Techniques to Scrape Website R: What Works in 2023?

25 Replies, 2378 Views

Yo, RSelenium is def heavy, but sometimes you gotta bite the bullet for dynamic stuff. If you’re looking for something lighter, check out htmlunit with R. It’s not as popular but works well for JS-heavy sites.

Also, for anti-scraping, try adding random delays between requests. I use Sys.sleep() with some randomness to avoid getting blocked.

And hey, if you’re scraping website R, don’t forget to check the site’s robots.txt. Some sites are cool with scraping as long as you respect their rules.

Messages In This Thread



Users browsing this thread: 1 Guest(s)