What’s the best way to screen scrape a web page without getting blocked? or How can I screen scrape a

20 Replies, 2238 Views

Lol @ “pray to the internet gods.” Been there.

For screen scrape web page stuff, I’ve had success with Puppeteer-extra + stealth plugin. Makes your bot look more like a real browser.

Also, don’t forget to respect robots.txt. Some sites will ban you faster if you ignore it.

Messages In This Thread



Users browsing this thread: 1 Guest(s)