The n8n webscraper is decent for static stuff, but dynamic sites? Eh, not so much out of the box.
You’ll def need a headless browser like Puppeteer or Playwright to handle JS-heavy pages. There’s a community node for Puppeteer that integrates with n8n, so check that out.
For rate limits, rotate your IPs or use proxies. Bright Data’s proxies work well, but they’re pricey. Free options like ScraperAPI can help too, but YMMV.
You’ll def need a headless browser like Puppeteer or Playwright to handle JS-heavy pages. There’s a community node for Puppeteer that integrates with n8n, so check that out.
For rate limits, rotate your IPs or use proxies. Bright Data’s proxies work well, but they’re pricey. Free options like ScraperAPI can help too, but YMMV.