Incorrect timestamp field type in for created Iceberg Table

Apache Iceberg
1

This is somewhat of a criticial bug.

Using dremio 26.0.0 OSS, when I create an iceberg table on top of AWS GLUE:

  • Dremio doesn’t have timstamp with timezone type, so the date cloumn is just a Timestamp
  • Parquet files created record a standard timestamp column -
  optional int64 field_id=6 created_at (Timestamp(isAdjustedToUTC=false, timeUnit=microseconds, is_from_converted_type=false, force_set_converted_type=false));
  • AWS Glue records a timestamp column (timestamptz are not supported)
  • However… the Iceberg metadata json file uses a timestamptz column.

When trying to read the table using pyiceberg, it just looks at the json metadata and fails due to type incompatibility between Timestamp and Timestamptz

You need to make ure timestamp columns in iceberg tables are properly recorded using the timestamp type

2

@sheinbergon Are you able to send “CREATE TABLE DDL” from Glue?