What's the best way to start crawling with Python for web data? or Need advice on crawling with Pytho

18 Replies, 1928 Views

Dude, just use `requests` + `BeautifulSoup` to start. Scrapy can wait.

For JS-heavy sites, `playwright` is my fav. Selenium is too clunky.

And yeah, watch out for bans. Use proxies and random delays.

PS: Real Python’s tutorial on crawling with Python is fire.

Messages In This Thread



Users browsing this thread: 1 Guest(s)