Wow, thanks for all the suggestions! I tried the `requests-html` one first since it looked easiest, and it actually worked for the site I was testing.
But now I’m curious—how do you guys handle sites that block scrapers? I got a 403 on one of them. Would rotating user-agents help, or do I need proxies for colab scrape images from website python?
Also, big thanks for the Selenium and Playwright tips—gonna try those next!
But now I’m curious—how do you guys handle sites that block scrapers? I got a 403 on one of them. Would rotating user-agents help, or do I need proxies for colab scrape images from website python?
Also, big thanks for the Selenium and Playwright tips—gonna try those next!