Changing the underlying schema for a table has really buggy behavior especially with select *

All sorts of weird things can happen if you change the type of a table where a dependent table is pointing at it with a select * statement. You get errors and the only way I have found to fix it is to completely save the query starting from a different SQL context. There is some cached metadata in the * operator that seems to be breaking dependent tables when you change the underlying schema (types or names).

I’ve seen a few things to indicate that this is… expected behavior?

I know that before saving a query it must complete a preview/run to establish columns and metadata. This also cements the list of parent dependencies and context information.

A SELECT * resolves to a column list behind the scenes in Dremio at the time of save.

The behavior you are bringing up though sounds similar to my past experiences with other database systems, where VIEW definitions need to be re-executed after a schema change to avoid breaking them. Not disagreeing that this couldn’t be automated in Dremio, or at least an offer in the UI to auto update.

Oooh, have a checkbox on the Metadata refresh tab in the UI, “Automatically rebuild dependent VDS when schema changes detected”, or something like that :smiley:

1 Like

@kprifogle,

#1 What i the error you are getting?
#2 What version of Dremio is this?

(1) A schema mismatch error either on parquet (if its reflected) or from calcite.
(2) 3.2

@kprifogle

Do you have a profile. Also 3.2 is a very old version, any reason you are still on that old version

Thanks
Bali