Hey, thanks for all the tips, everyone!
I tried slowing down requests + rotating user agents, and it’s way better now. Still getting some blocks though—might need to invest in better proxies like some of you suggested.
Quick Q: Anyone know if Cloudflare sites are just impossible to scrape jobs from the internet? Or is there a workaround?
Also, ParseHub looks interesting—gonna test that next. Appreciate the help!
I tried slowing down requests + rotating user agents, and it’s way better now. Still getting some blocks though—might need to invest in better proxies like some of you suggested.
Quick Q: Anyone know if Cloudflare sites are just impossible to scrape jobs from the internet? Or is there a workaround?
Also, ParseHub looks interesting—gonna test that next. Appreciate the help!
