Need help building a Python program to scrape a website – any tips or libraries to use? or What’s the

25 Replies, 2138 Views

If you’re building a python program to scrape website data, check out `httpx` for async requests—way faster than `requests` for large jobs.

Also, Cloudflare sucks. If a site uses it, you might need `undetected-chromedriver` with selenium.

Here’s a snippet:
```python
import httpx
response = httpx.get(url, headers=headers)
```

Messages In This Thread



Users browsing this thread: 1 Guest(s)