OP reply:
Wow, thanks for all the suggestions! Didn’t expect so many tools I hadn’t heard of.
Tried `pandas` with `error_bad_lines=False` and it saved me a ton of headache already.
Anyone have experience with `Great Expectations` vs `Trifacta`? Wondering which is better for long-term maintenance.
Also, how do you guys handle encoding issues? Still running into weird characters in some files...
Wow, thanks for all the suggestions! Didn’t expect so many tools I hadn’t heard of.
Tried `pandas` with `error_bad_lines=False` and it saved me a ton of headache already.
Anyone have experience with `Great Expectations` vs `Trifacta`? Wondering which is better for long-term maintenance.
Also, how do you guys handle encoding issues? Still running into weird characters in some files...
