Wow, thanks everyone! Didn’t expect so many tips.
Tried switching to `lxml` and added headers—way fewer errors now. Still kinda slow, but the `time.sleep(1)` hack helps.
One follow-up: how do you handle login pages with beautiful soup jupyter notebook? Do you just use `requests.post()` or is there a better way?
Also, gonna check out those tutorials y’all dropped. Appreciate it! 🙌
Tried switching to `lxml` and added headers—way fewer errors now. Still kinda slow, but the `time.sleep(1)` hack helps.
One follow-up: how do you handle login pages with beautiful soup jupyter notebook? Do you just use `requests.post()` or is there a better way?
Also, gonna check out those tutorials y’all dropped. Appreciate it! 🙌