Has Anyone Used the OpenAI Crawling API for Web Data Extraction? or What Are the Best Use Cases for t

18 Replies, 1116 Views

Biggest limitation? CAPTCHAs. The openai crawling api just gives up if it hits one.

Also, some sites load content via API calls, and it doesn’t execute JS, so you’ll miss a ton. For that, try Selenium or Playwright.

Messages In This Thread



Users browsing this thread: 1 Guest(s)