Yeah, I’ve tried to scrape data using llm a few times, and it’s… interesting.
I fed raw HTML into GPT-4 and asked it to extract specific tables or lists. Worked okay for simple stuff, but yeah, the hallucinations are real.
For better results, I pre-processed the HTML with something like BeautifulSoup to clean it up first. Less noise = better accuracy.
Still, traditional scraping is way more reliable for bulk work. LLMs are fun for quick, one-off extractions though.
I fed raw HTML into GPT-4 and asked it to extract specific tables or lists. Worked okay for simple stuff, but yeah, the hallucinations are real.
For better results, I pre-processed the HTML with something like BeautifulSoup to clean it up first. Less noise = better accuracy.
Still, traditional scraping is way more reliable for bulk work. LLMs are fun for quick, one-off extractions though.