How to scrap data from web with Golang – Any tips or best practices? or What’s the best way to scrap

18 Replies, 782 Views

Colly’s cool, but if you’re scraping JS-heavy sites, you might need a headless browser like chromedp. Golang’s not the best for this, but it works.

Avoiding blocks: Use rotating IPs and set realistic delays. Tools like ProxyMesh can help.

Speed: Goroutines, but don’t overload the server. Be polite—or they’ll ban you forever.

Messages In This Thread



Users browsing this thread: 2 Guest(s)