Parquet & Dremio - no best friends?

1

From time-to-time we face issues with Dremio not being able to read parquet files that other tools (like spark, pandas) can handle easily.

And the bad thing is, that the error messages we got from Dremio aren’t helpful at all. This also true for other errors.

e.g.: “Error in parquet reader (complex). Message: Failure in setting up reader Parquet Metadata:”

My wish…

  1. Please set parquet compatibility as a higher priority for development.
  2. Please provide error messages that speak to your developer AND the Dremio users.

Many thanks
Thomas

2

Hi @thomasz,

Thanks for reporting this. Is it possible to share some examples of these files?

3

Hi @ben , I can share parquet files that Dremio can not read. What is the best way to send them to you ?

4

Hi @dfleckinger If they do not contain any private content, can you please upload them to this thread? I will create internal tickets as required.

5

Hi @Praveen_Kumar, they do contain private content.
I have sent them by email to @ben and @balaji.ramaswamy privately in the thread Regression in parquet reader in version 3.3.1