Hey! I’ve tried scrapping linkedin screaming frog before, and yeah, LinkedIn’s anti-scraping is no joke. I got blocked within like 2 hours lol.
What worked for me was using rotating proxies and slowing down the crawl speed. Also, make sure to randomize the user-agent strings.
If you’re not super tied to Screaming Frog, you might wanna check out Octoparse or Scrapy. They’re a bit more flexible for scraping LinkedIn.
Good luck!
What worked for me was using rotating proxies and slowing down the crawl speed. Also, make sure to randomize the user-agent strings.
If you’re not super tied to Screaming Frog, you might wanna check out Octoparse or Scrapy. They’re a bit more flexible for scraping LinkedIn.
Good luck!