Dremio not returning records from a parquet file

1

Hello Dremio,

My company uses a Dremio version 22.1.7, we have been reading parquets from our s3 bucket,

I loaded parquet files from a dataframe with this setings below.
df.to_parquet(path=file_path_name, index=False,compression='snappy')

Dremio was able to read the parquet in the data settings as usual

image
image1213×725 55.2 KB

but when I created a PDS on the file , no records is been returned.

image
image1647×881 58.8 KB

2

@iskidet01 Could you try clicking the Run button to see if a full run returns results?

3

I haved resolved the issue ,I basically have to covert all schema to string using
df=df.applymap(str) before rewrting to parquet, it appears dremio has issue dealing with variant datatype in parquet format