+- Proxy Community (https://proxycommunity.com/forum)
+-- Forum: Technical Community Support (https://proxycommunity.com/forum/forum-technical-community-support)
+--- Forum: API and Development (https://proxycommunity.com/forum/forum-api-and-development)
+--- Thread: [b]"What's the best Python XML parser for handling large files efficiently?"[/b] or [b]"How do you choose the righ (/thread-b-what-s-the-best-python-xml-parser-for-handling-large-files-efficiently-b-%0A%0Aor-%0A%0A-b-how-do-you-choose-the-righ)
“” - HiddenX_88 - 31-03-2025
If you’re stuck with a giant XML file, maybe don’t parse it all at once?
`lxml` + `iterparse` is the classic move, but `xml.sax` is lighter.
Or, if you’re lazy (like me), throw it into a database (PostgreSQL has XML support) and query it from there.
“” - cloakSurfer99 - 31-03-2025
lxml is still king for speed, but yeah, memory can be a pain.
Try `ElementTree.iterparse()` with manual cleanup—it’s not *as* fast as lxml, but gentler on RAM.
Or, if you’re feeling fancy, `pyarrow` + `pandas` for tabular XML data.