Hey! I’ve been down this road before, and it’s definitely tricky. One thing I’d recommend is using Octoparse if you’re not super comfortable with coding. It’s a no-code tool that lets you scrape data visually, and it’s great for structured data like tables.
Also, for APIs, check out Public APIs GitHub repo. It’s a goldmine for finding APIs, and you might not even need to scrape.
If you do end up scraping, make sure to handle CAPTCHAs. Some sites throw them at you like crazy.
Also, for APIs, check out Public APIs GitHub repo. It’s a goldmine for finding APIs, and you might not even need to scrape.
If you do end up scraping, make sure to handle CAPTCHAs. Some sites throw them at you like crazy.