Hey! I’ve been using GoQuery for static pages and Rod for dynamic content. Rod’s a bit tricky to set up, but once you get it working, it’s amazing.
For rate-limiting, I usually start with a 2-second delay and adjust based on the site’s response. If I get blocked, I switch to rotating proxies.
Also, don’t forget to check out Scrapy (Python) for inspiration. It’s not Golang, but their approach to scraping is top-notch.
For rate-limiting, I usually start with a 2-second delay and adjust based on the site’s response. If I get blocked, I switch to rotating proxies.
Also, don’t forget to check out Scrapy (Python) for inspiration. It’s not Golang, but their approach to scraping is top-notch.