I’ve done this before! For python web scrape an article github, BeautifulSoup works fine, but you gotta respect the rate limits.
Pro tip: Use `session` in requests to persist cookies and headers.
Also, if you’re scraping a lot, check out `selenium` for JS-heavy pages.
Pro tip: Use `session` in requests to persist cookies and headers.
Also, if you’re scraping a lot, check out `selenium` for JS-heavy pages.
