Hi,
Querying with SQL a Dremio Table from parquets in S3, I am getting an error reading TIMESTAMP columns:
Unable to coerce from the file’s data type “int64” to the column’s data type “timestamp”…
Describing the parquet file in DBeaver, data is read properly and column types are defined like: TIMESTAMP WITH TIME ZONE. QGIS is reading the files properly as well.
What do I need to configure when promoting the Table from the S3 folder of parquets?
Please, can you help me?
part0.zip (2.5 KB)
Thanks in advance
@ahuarte47
Parquet-tools tells a codec is missing. How was this file genearted and with which options?
% java -jar parquet-tools-1.9.0.jar schema ~/Downloads/part0.parquet
can not read class org.apache.parquet.format.FileMetaData: Required field 'codec' was not present! Struct: ColumnMetaData(type:INT64, encodings:[RLE_DICTIONARY, PLAIN, RLE], path_in_schema:[_transaction_ts], codec:null, num_values:66, total_uncompressed_size:95, total_compressed_size:113, data_page_offset:35, dictionary_page_offset:4, statistics:Statistics(max:EC 90 D1 36 42 42 10 18, min:EC 90 D1 36 42 42 10 18, null_count:0), encoding_stats:[PageEncodingStats(page_type:DICTIONARY_PAGE, encoding:PLAIN, count:1), PageEncodingStats(page_type:DATA_PAGE, encoding:RLE_DICTIONARY, count:1)])
Hi, thank you very much for your help.
I am using the FME Apache Parquet Writer (Apache Parquet Reader/Writer) to generate this output.
It seems there is no anything to configure this. Maybe, is it a bug?
Regards
Rewriting the parquet with duckdb with COPY INTO… TO… the problem is go out and Dremio loads the new parquet properly.
@ahuarte47 Good to hear that,