Reflections Dropped by Dremio even with Refresh policy set to Never

We refreshed the dremio cluster and logged in and found all of the reflections had been removed by dremio. Here is a profile for an example query that $dremio triggered to remove a reflection. Do you have any idea why this happened? This is in spite of Never Expire being set on the Source. (3.5 KB)

Also I noticed on a seperate occasion that the source Reflection settings had been updated without anyone doing so. Is it possible some of these settings are cached locally so rolling the cluster would clear them out?

Hi @kprifogle

Can you post the screenshot of the reflection refresh policy screenshot on the source as well as the dataset.

See if you have any API calls managing the reflections outside of Dremio UI



Reflection Refresh policy is set to “Never Refresh” and “Never Expire”, and incremental refresh is turned on.

I dont have any external api calls managing the reflections outside dremio. Also the $dremio user deleted the reflection so it seems like it was part of a standard daily clean up.



never refresh and never expire work only on reflections that have their setting at default, like attached image. It seems like changing to “Incremental Refresh” does override “Never expire”. Do you also see
“REFRESH REFLECTION” jobs on this dataset?