Cascading Reflection Refresh

1

Hello,

I read that when the reflection of a parent dataset is refreshed, it would refresh the reflections of the children. I have a source dataset based on csv files in azure. The source dataset has a raw reflection. I am also using dbt to create a staging version of that source dataset which also has a raw reflection. I assume that constitutes a parent/child relationship.

When I refresh the source reflection via API, it does get refreshed. However, I do not see anything in the job log for the child staging dataset. When I navigate to the reflection details via the UI, it also shows the older reflection.

I am using an old (v4) version of Dremio. Would you expect to see the child (staging) reflection job in the job list based on its parent (source) reflection being refreshed?

Thanks,
Drew

2

@drew

Are the 2 datasets PDS’s or one of them is a VDS? are you able to select from both datasets limit 1 and send over the profiles?

3

The parent dataset is a PDS. The child datasets are VDS. Both have reflections

4

I would need to see the profiles of the 2 datasets and I can see if something is missing. Say you have a PDS called “emp” and 2 VDS’s “emp_vds1” and “emp_vds2”. Both are just select * from emp, say both have reflections, if you run the refresh catalog API on the PDS, both the reflections will get refreshed and both VDS will see new data.