What’s the best way to scrape jobs from the internet without getting blocked? or How can I efficientl

18 Replies, 956 Views

Hey, thanks for all the tips, everyone!

I tried slowing down requests + rotating user agents, and it’s way better now. Still getting some blocks though—might need to invest in better proxies like some of you suggested.

Quick Q: Anyone know if Cloudflare sites are just impossible to scrape jobs from the internet? Or is there a workaround?

Also, ParseHub looks interesting—gonna test that next. Appreciate the help!

Messages In This Thread



Users browsing this thread: 1 Guest(s)