Lol @ “pray to the internet gods.” Been there.
For screen scrape web page stuff, I’ve had success with Puppeteer-extra + stealth plugin. Makes your bot look more like a real browser.
Also, don’t forget to respect robots.txt. Some sites will ban you faster if you ignore it.
For screen scrape web page stuff, I’ve had success with Puppeteer-extra + stealth plugin. Makes your bot look more like a real browser.
Also, don’t forget to respect robots.txt. Some sites will ban you faster if you ignore it.
