I feel you on the semi-structured data struggle. I’ve been using AWS Glue to catalog and transform data, and it’s been pretty solid. It’s not perfect, but it handles inconsistencies better than most tools I’ve tried.
Also, Athena is great for querying semi-structured data directly from S3. It’s serverless, so you don’t have to worry about managing infrastructure.
Also, Athena is great for querying semi-structured data directly from S3. It’s serverless, so you don’t have to worry about managing infrastructure.
