Need Help: How to Scrape Data from Web with Golang Effectively?

11 Replies, 1365 Views

I feel you on the outdated tutorials thing! I struggled with that too. I ended up using `goquery` for static pages and `chromedp` for JS-heavy ones.

For pagination, I just wrote a recursive function to keep scraping until there’s no “next” button. It’s not perfect, but it works.

Also, don’t forget to respect the site’s `robots.txt` file!

Messages In This Thread



Users browsing this thread: 1 Guest(s)