What's the best way to scrape cards data using Python? or How can I efficiently scrape cards data wit

20 Replies, 1918 Views

Hey! For scrape cards python, Selenium is solid but heavy. Playwright is faster and handles JS better imo.

Try this for lazy-loading:

```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("your_url")
page.evaluate("window.scrollTo(0, document.body.scrollHeight)") # scrolls to load more
cards = page.query_selector_all(".card-class")
```

Also, rotate user-agents + proxies to avoid bans. Free proxies? Check free-proxy-list.net.

Messages In This Thread



Users browsing this thread: 1 Guest(s)