Yo, dynamic content is the WORST. I feel your pain. One thing that worked for me was using `rvest` with `xml2` to parse the HTML after the page fully loads. Sometimes, you can even inspect the network requests in your browser’s dev tools and mimic them with `httr` to get the data directly.
For pagination, I’d suggest adding delays between requests (`Sys.sleep()`) to avoid getting blocked. Also, check out `polite` package—it’s great for respectful scraping.
Hope this helps!
For pagination, I’d suggest adding delays between requests (`Sys.sleep()`) to avoid getting blocked. Also, check out `polite` package—it’s great for respectful scraping.
Hope this helps!