From time-to-time we face issues with Dremio not being able to read parquet files that other tools (like spark, pandas) can handle easily.
And the bad thing is, that the error messages we got from Dremio aren’t helpful at all. This also true for other errors.
e.g.: “Error in parquet reader (complex). Message: Failure in setting up reader Parquet Metadata:”
- Please set parquet compatibility as a higher priority for development.
- Please provide error messages that speak to your developer AND the Dremio users.