What’s the best way to screen scrape a web page without getting blocked? or How can I screen scrape a

25 Replies, 1408 Views

Hey! Newbie here too. I found that screen scraping web pages works best if you throttle requests. Like, 1 request every 3-5 seconds.

Tools? Start with Requests + BeautifulSoup. If it’s JS-heavy, Selenium or Puppeteer.

Proxies help, but free ones are sketchy. I use Luminati when I’m feeling fancy.

Messages In This Thread



Users browsing this thread: 1 Guest(s)