Pro tip: if the site uses lazy loading, sometimes you can just inspect the network tab and find a JSON endpoint with all the image URLs.
For colab scrape images from website python, you can skip the HTML parsing entirely and just `requests.get()` that JSON. Saves a ton of time!
Example:
```python
import requests
import json
url = "https://site.com/api/images" # found in network tab
data = requests.get(url).json()
for img in data['images']:
# download logic here
```
Way cleaner than dealing with Selenium IMO.
For colab scrape images from website python, you can skip the HTML parsing entirely and just `requests.get()` that JSON. Saves a ton of time!
Example:
```python
import requests
import json
url = "https://site.com/api/images" # found in network tab
data = requests.get(url).json()
for img in data['images']:
# download logic here
```
Way cleaner than dealing with Selenium IMO.
