What's the best way to parse HTML/XML with lxml in Python? or How do you handle large XML files effic

14 Replies, 1822 Views

lxml is fast because it's basically a Python wrapper around libxml2, which is a battle-tested C library. No surprises there.

But have you tried `parsel`? It's what Scrapy uses, and it's *almost* as fast as lxml python but with a nicer API. Plus, it plays well with Scrapy if you're into large-scale scraping.

For benchmarks, check out this repo: [github.com/lorien/html_parser_benchmark](https://github.com/lorien/html_parser_benchmark). It's a bit outdated but still useful.

Messages In This Thread



Users browsing this thread: 1 Guest(s)