How can I use Beautiful Soup in a Jupyter Notebook for web scraping? or Best practices for running Be

16 Replies, 1911 Views

Hey! First off, make sure you’re importing `requests` AND `BeautifulSoup` correctly. Like this:
```python
from bs4 import BeautifulSoup
import requests
```

Also, try `lxml` instead of `html.parser`—it’s way faster and less prone to weird errors.

For slow scraping, maybe add some delays with `time.sleep(1)` between requests? Or try `requests.Session()` to keep the connection alive.

Here’s a quick example that works in my beautiful soup jupyter notebook:
```python
url = "https://example.com"
response = requests.get(url)
soup = BeautifulSoup(response.text, 'lxml')
print(soup.prettify())
```

Hope that helps!

Messages In This Thread



Users browsing this thread: 1 Guest(s)