Should You Process JSON One by One or All at Once? What’s the Best Approach?

18 Replies, 1268 Views

Wow, thanks everyone for the awesome replies! I didn’t expect so many helpful insights.

I think I’ll try processing JSON one by one for my current project since the dataset is pretty large. I’ll give `ijson` a shot—seems like the way to go for memory efficiency.

Also, thanks for the `jq` and `jsonlines` recommendations. I’ll definitely check those out for smaller tasks.

One quick follow-up: has anyone tried combining both methods? Like processing chunks of JSON at a time instead of one by one or all at once? Just curious if that’s a thing.

Thanks again, y’all are the best!

Messages In This Thread



Users browsing this thread: 1 Guest(s)