Incompatible types in file and table

1

We get this error when trying read a specific table on our data lake.
Field [DATE_REMOVED] has incompatible types in file and table. Type in fileschema: [DATE_REMOVED: INTEGER], type in tableschema: [date_removed: VARCHAR]
The Hive table we are reading is built on a underlying parquet file on HDFS.

Not sure what we need to change to allow Dremio to read the data. We have a lot of Hive tables with a similar error.

Does the file in this case refer to the parquet file or the underlying data file?

2

@Corey

The column DATE_REMOVED in the Parquet file has type INTEGER while in the Hive table definition is VARCHAR. Currently neither Hive nor Dremio would support this

Are you able to change the table definition? Are all the files containing this column are of type INTEGER? If you decide to change the data type on Dremio, remember to refresh metadata via Dremio

https://docs.dremio.com/sql-reference/sql-commands/datasets.html