How can I efficiently scrape websites using bs4 Python? or What are the best practices for parsing HT

14 Replies, 1468 Views

Dude, I feel you. bs4 python can be tricky at first.

For your price issue, maybe the class isn't *just* 'price'. Try:
```python
soup.find_all('div', class_=lambda x: x and 'price' in x)
```
Or use `attrs={'class': 'price'}`.

Some sites hide data in `data-` attributes too. DevTools is your best friend here!

Messages In This Thread



Users browsing this thread: 2 Guest(s)