Struggling with R scraping? Any tips for handling dynamic content and pagination?

7 Replies, 1723 Views

Wow, thanks so much for all the tips, y’all! I didn’t expect so many helpful replies. I tried using `httr` to mimic network requests, and it worked for some of the dynamic content—total game-changer!

I’m still struggling with pagination, though. I tried the `while` loop idea, but it keeps breaking after a few pages. Maybe I’ll give `furrr` a shot for parallel processing.

Also, someone mentioned `chromote`—never heard of it before, but I’ll definitely check it out. Thanks again, everyone! This community is awesome.

Messages In This Thread



Users browsing this thread: 1 Guest(s)