External_query() reflection refresh schedule - what is it?

I have Virtual Data Set that uses external_query() to make sure Snowflake does the heavy lifting rather than Dremio downloading an entire table and then doing some simple transforms and groupings.

When would a Reflection on this Virtual Data Set be refreshed? According to the “Reflection Refresh” configs on the data source used by external_query() or something else? Some observations follow:

I can manually refresh the reflection on the Virtual Data Set by disabling the reflection, hitting Save, then enabling the reflecting and hitting Save.

The data source used by external_query() has “Reflection Refresh” configs, but they always reset to “Refresh every 1 Hour”, even if I change it to a once-per-day schedule and hit Save. Am I missing something that might be overriding that config?

I wish I could just schedule a reflection refresh on the Virtual Data Set directly, regardless of any Physical Data Set refresh schedules.

I’ve seen the following discussions, and it seems like a “refresh Virtual Data Source reflections” feature would be a good one. I’m guessing it’s one of those things that isn’t that simple, but I’d like to vote for it.

Yes, that’s how it works. If your Source Settings → Reflection Refresh → Refresh Settings is set to every hour, then the reflection should refresh every hour.

If you are not able to change that to 24 hour, that seems like a bug.

You could also script the reflection creation and deletion with SQL.



Thanks for the confirmation. Reflections that use external_query() are refreshed according to the reflection refresh schedule of the data source itself (rather than any specific table inside the data source).

What is the best way to report a bug? Just create a new post here? edit: I just created a new post about the data source reflection refresh settings not saving as expected.