BeautifulSoup is a beast, but yeah, it can be a pain. For beautifulsoup html table to json, I’ve found that cleaning the HTML first with something like `html5lib` makes a huge difference. It standardizes the structure, so you don’t have to deal with as many edge cases.
Also, if you’re working with large datasets, consider using `jsonlines` instead of `json.dumps`. It’s more memory-efficient for big files.
Also, if you’re working with large datasets, consider using `jsonlines` instead of `json.dumps`. It’s more memory-efficient for big files.