Thanks for all the suggestions, folks! I tried Playwright based on your advice, and it’s way faster than Selenium. Still running into some captchas, but rotating proxies seem to help.
I’m also looking into their API for long-term use. Anyone know if they’re responsive to requests for higher limits? Also, has anyone tried combining the API with scraping for more frequent updates?
I’m also looking into their API for long-term use. Anyone know if they’re responsive to requests for higher limits? Also, has anyone tried combining the API with scraping for more frequent updates?
