I have a parquet file that has different columns, one of the columns is of type Datetime. When I query the parquet file it says “No Results in Preview Sample”. When I query individual columns (other than the datetime) I am able to view the values, however, when I query the datetime column, it again returns “No Results in Preview Sample”. However, my datetime column seems to fine as shown in the test python script. May I know why Dremio returns no results when there are values? Thank you
I think dremio has an issue reading parquet format 2.6 - which is now the default for pyarrow v9+. You can pass parquet_format=“1.0” when writing the parquet file and you’ll get the old default behavior- which dremio should be happier with.
1 Like
Changing the version worked, but why is it like that, shouldn’t we be able to use the latest parquet version? Can someone from Dremio please clarify this a bit? It is happening only for time-series data.
I actually tried adding an index column and once I added that it started working. But it doesnt always work in this case, could Dremio help me understand why this is happening as well? Thank you!