What's the best way to build a Python web crawler for scraping large sites? or How can I optimize my

23 Replies, 2068 Views

Wow, didn’t expect so many gems here! Def gonna try `requests-html` and `playwright`—Selenium’s been killing my vibe.

Also, *logging*. Why didn’t I think of that? Already saved me once today.

One Q: Anyone got a fav proxy service that won’t bankrupt me? The free ones are… yikes.

(And thx for the Scrapy push—might dive in after all.)

Messages In This Thread



Users browsing this thread: 1 Guest(s)