Struggling with R scraping: How do you handle dynamic content and pagination?

14 Replies, 1184 Views

Wow, thanks for all the tips, everyone! I tried RSelenium with `waitForElement()` and it’s working way better now. Still struggling a bit with pagination, but the `purrr::map()` suggestion sounds promising—gonna give that a shot next.

Also, the `html_session()` tip was a game-changer for simpler pages. I’ll check out that tutorial too.

One quick follow-up: anyone know how to handle CAPTCHAs while scraping? That’s my next hurdle lol. Thanks again, y’all are legends! 🙌

Messages In This Thread



Users browsing this thread: 1 Guest(s)