Thanks for all the suggestions, everyone! I’ve been trying out some of the tools mentioned, like ScrapingBee and Puppeteer, and they’ve definitely helped reduce the *scraping failures unknown*.
Still getting some 403s, though. Has anyone had luck with specific residential proxies? I’m thinking of giving Oxylabs a shot, but they’re a bit pricey.
Also, anyone know if sites are using more AI-based detection now? Feels like they’re getting smarter every day.
Thanks again for the help!
Still getting some 403s, though. Has anyone had luck with specific residential proxies? I’m thinking of giving Oxylabs a shot, but they’re a bit pricey.
Also, anyone know if sites are using more AI-based detection now? Feels like they’re getting smarter every day.
Thanks again for the help!