Querying a dataset onboarded from an AWS Glue catalog.
The field ‘xxx’ is defined as a TIMESTAMP in the catalog, and persisted as a BIGINT in the underlying Parquet files.
Attempting to define where predicates in the form
AND xxx >= TIMESTAMP ‘2020-12-21 14:00:00’
AND xxx < TIMESTAMP ‘2020-12-21 14:15:00’
results in a parsing error
Field [xxx] has incompatible types in file and table. Type in fileschema: [xxx: BIGINT], type in tableschema: [start_date_utc: TIMESTAMP]
Fields defined as DATE in the catalog and persisted as BIGINT have no issue.
AND yyy >= DATE ‘2020-12-21’
AND yyy < DATE ‘2020-12-22’