Table does not include column identified for incremental update of name '$_dremio_$_update_$'

In case anyone else is facing this error when building a Raw Reflection for a Physical Dataset:

Table does not include column identified for incremental update of name '$_dremio_$_update_$'.

It appears to occur when you have selected the option “Incremental update based on new files” for a Physical Dataset that uses partitioned Parquet files and you don’t select the partition fields “dir0” (and/or “dir1”, “dir2”, etc) for the Raw Reflection.

I resolved this by removing the Raw Reflection, modifying the Reflection Settings’ Refresh Method to use Full Update.
Screen Shot 2022-02-15 at 3.08.47 PM

And then turning on the Raw Reflection with the dir0 fields, letting it build and then re-enabling the Incremental update option.

Hi @phillip Thanks for the update, will review this one