Invalid millis for timestamp value when using a raw reflection

1

In Dremio-Cloud (EU), I’m getting this error:

GandivaException: Invalid millis for timestamp value 2023-01-01 00:00:00.000000

when I query a dataset that has an applicable raw reflection and use this value in the “where” clause:
2023-01-01 00:00:00.000000
In this case, I am querying a view stored in the Arctic catalog that is pointed at a Snowflake table. So it’s a dremio view of a Snowflake Datasource table.

Here is the full query:

SELECT
*
FROM analytics."my_table"."Sessions"
WHERE "CREATEDON" >= '2023-01-01 00:00:00.000000'

The error goes away if I reduce the number of decimal places to 3 or fewer:
2023-01-01 00:00:00.000

Additional Observations

If I remove the raw reflection and run the query with the full 6-digit value, the error does not happen.

If I remove the raw reflection and add an aggregate reflection, the error does not happen.

It seems to be a problem when a raw reflection has been created for the dataset and that reflection is being used.

I get this error when I run the query within the Dremio Web UI, and I get it when I run the query via Superset. I expect it would return this error when run from any client.

In the past, I had seen kind of like this in Superset for queries submitted to dremio-oss via Superset until I used this sqlalchemy_dremio fork from lenoyjacob as part of my Superset deployment: git+https://github.com/lenoyjacob/sqlalchemy_dremio.git. Just mentioning the closest thing I have seen.

What can I do to work around this?

2

@capnjosh Currently Dremio only supports up to Milli seconds. I am surprised the query works when not reflected, can you please send both the reflected profile (with Gandiva error) and the one without the reflection that works

3

I tried to reproduce it just now, but it’s working just fine.

I will update here if I see it happen again, and I’ll provide profile.