What's the best way to start crawling with Python for web data? or Need advice on crawling with Pytho

18 Replies, 1718 Views

"Need advice on crawling with Python—any tips for beginners?"

Hey everyone!

So I’m kinda new to crawling with Python and wanna get started, but there’s *so* much info out there it’s a bit overwhelming.

What’s the best way to begin? Like, should I start with `requests` + `BeautifulSoup` or jump straight to `Scrapy`? Also, how do you even handle sites with dynamic content? Selenium seems heavy, but idk if there’s a lighter alternative.

And uh… any gotchas I should watch out for? Don’t wanna accidentally get banned lol.

Thanks in advance! 🙏

(ps. if you’ve got favorite tutorials or libs, drop ‘em below!)

Messages In This Thread
What's the best way to start crawling with Python for web data? or Need advice on crawling with Pytho - by - 08-05-2024, 02:40 AM



Users browsing this thread: 1 Guest(s)