[b]"Need help building a Python program to scrape a website – any tips or libraries to use?"[/b] or [b]"What’s the

18 Replies, 497 Views

Speed tips for your python program to scrape website data:
- Use `concurrent.futures` for parallel requests (but don’t DDoS the site lol).
- Scrapy’s built-in concurrency is solid.
- Avoid selenium unless you *need* JS rendering—it’s slow af.

For blocks, some sites freak out if you don’t accept cookies. Try adding that to your headers.

Messages In This Thread



Users browsing this thread: 1 Guest(s)