Dremio 3.2.4 INVALID_DATASET_METADATA ERROR Unexpected columns

1

I have a mysql database and a set of parquet files. The parquet data is curated and joined to the mysql database to create a VDS that has a raw reflection (referenced later).

Then I join this VDS with the mysql database for dimension details. This works but when I try to create a reflection on the joined dataset I get an INVALID_DATASET_METADATA error. I tried various refresh metadata and forget metadata command and the error still popped up.

On a whim I removed the reflection on the original VDS and tried to generate the dimension joined reflection and then it worked. If I add the raw reflection back to the original VDS the joined VDS reflection fails again.

I am attaching the profile if that helps.

b8125603-3033-4502-920d-2d37c39e5277.zip (271.2 KB)

2

Hi @mlilius

This looks like a known issue we are working on a fix. When fixed we should have it in the release notes of the fixed version and will update this thread

Thanks
@balaji.ramaswamy

3

Thanks for the update. I am wondering what the root cause is so I can try to avoid it for now. It keeps popping up for me.

Any insights would be appreciated.