How to use Python to get URL and ignore 404 errors?

18 Replies, 1411 Views

Hey there! If you're doing a lot of scraping and want a more robust solution, check out the `httpx` library. It’s like `requests` but async and handles errors gracefully.

For python get url ignore 404, you can do:
```python
import httpx

async def fetch(url):
async with httpx.AsyncClient() as client:
try:
response = await client.get(url)
return response
except httpx.HTTPStatusError:
print(f"404 for {url}, skipping!")
```
This is great for large-scale scraping.

Messages In This Thread



Users browsing this thread: 1 Guest(s)