I am trying to query a parquet file which was generated using C# Parquet.Net library, the metadata retrieval goes for ever and fails…
The same parquet file can be read from spark 3.1.2. The data is pulled from SQL Server and written to parquet file using the above mentioned library.
Any help to understand the issue would be appreciated.
Dremio version : Community Edition Build 24.0.0
We’d need at least some details of the issue…
errors which come
Failed to decode column ::varchar ( this error comes most of the time )
failed to fetch ( this comes when parquet is converted to PDS )
The parquet file has just 1 field (string datatype) and 5 rows.
@manojg Can you please provide the job profile of the failed job? Does this parquet file contain sensitive data? If not, are you able to send a sample. If yes, then can you please send me the footer info using the parquet-tools meta <parquet-file command?