Playwright’s my jam for scraping. The built-in waits are a lifesaver, and it’s way less flaky than headless chrome with Puppeteer.
For avoiding blocks, I do:
- Rotate user-agents
- Mimic scroll behavior
- Use session cookies
Oh, and don’t forget `--disable-dev-shm-usage` if you’re running in Docker.
For avoiding blocks, I do:
- Rotate user-agents
- Mimic scroll behavior
- Use session cookies
Oh, and don’t forget `--disable-dev-shm-usage` if you’re running in Docker.
