[b]"What's the best way to parse HTML/XML efficiently with lxml Python?"[/b] or [b]"How do you handle large XML fi

20 Replies, 804 Views

For big XML, `iterparse()` + `clear()` is the golden combo. Otherwise, memory goes brrrr.

lxml Python’s `html` module is lenient, but `etree` is *strict* and faster. Pick your poison.

Also, `lxml.html.clean.Cleaner()` is great for sanitizing scraped HTML.

---
Wow, thanks for all the killer tips! I tried `iterparse()` with `clear()` on a giant XML dump, and it’s *way* better than loading it all at once.

One follow-up: Anyone got a trick for handling malformed HTML that still lets lxml Python stay fast? The `html` module helps, but sometimes it’s still a crawl.

Also, big shoutout for the `remove_blank_text` tip—saved me a headache!



Users browsing this thread: 1 Guest(s)