Cannot query parquet files produced from Python's pyarrow to_parquet()

1

Hi everyone,

I produced some parquet files by using to_parquet(). The engine used is pyarrow ==9.0.0, I then put the files onto HDFS and promote the files in Dremio. Dataset preview shows the data correctly. But once the files are promoted and I tried to query the dataset, i get no result return.

Am I missing something?

best regards,

2

@borasy Are you able to send the profile of the job and the server.log from the coordinator when this query ran

3

I think this issue has the same root cause as DateTime column returning no results - #2 by craustin

Specifying parquet_format=“1.0” should fix this for now.

4

Thanks for your response.