Hey there! I’ve been using a combo of BeautifulSoup and requests to scrape reddit page, but yeah, the rate limits are a pain.
One thing that’s helped me is rotating user agents and using residential proxies. It’s a bit of a setup, but it’s worth it if you’re scraping reddit page regularly. I’ve heard good things about Bright Data for proxies, but haven’t tried it myself yet.
One thing that’s helped me is rotating user agents and using residential proxies. It’s a bit of a setup, but it’s worth it if you’re scraping reddit page regularly. I’ve heard good things about Bright Data for proxies, but haven’t tried it myself yet.