How to use Colab to scrape images from a website with Python? or Best way to scrape images from a web

16 Replies, 1364 Views

Pro tip: if the site uses lazy loading, sometimes you can just inspect the network tab and find a JSON endpoint with all the image URLs.

For colab scrape images from website python, you can skip the HTML parsing entirely and just `requests.get()` that JSON. Saves a ton of time!

Example:

```python
import requests
import json

url = "https://site.com/api/images" # found in network tab
data = requests.get(url).json()
for img in data['images']:
# download logic here
```

Way cleaner than dealing with Selenium IMO.

Messages In This Thread



Users browsing this thread: 1 Guest(s)