Thanks so much for all the suggestions, everyone! I tried using a proxy service like Bright Data, and it seems to be working better now. Still getting the occasional cloudscraper 403 forbidden error, but it’s way less frequent.
I also switched to using Puppeteer for some of the trickier pages, and it’s been a game-changer.
Quick question though—does anyone know if there’s a way to detect if a site is using Cloudflare specifically? Just curious if that’s part of the issue. Thanks again!
I also switched to using Puppeteer for some of the trickier pages, and it’s been a game-changer.
Quick question though—does anyone know if there’s a way to detect if a site is using Cloudflare specifically? Just curious if that’s part of the issue. Thanks again!