I generated a VDS that queries several other VDS and is not configured to use a reflection, because I need fresh data on every result for this particular dataset.
I checked consistency between these view and a raw dataset queried directly from the source RDMS, and I detected some discrepancy. Checking the job that gathered the data from the VDS I realized it is using a reflection from a view that is not being called within the VDS definition.
I thought that reflections were only used between tables involved in the view since it doesn’t make any sense that Dremio used a reflection between tables not explicitly referenced.
Is this behavior normal? Is there a way to avoid it?
@jlcornejo Dremio lists all reflections that are on any dataset that are part of the query (can also be a VDS that the downstream VDS refers to) or logically equivalent has a reference. As you can see it was not matched so would not have used up any planning time.
that’s the problem, at no time do I use the vds that have a reflection.
- Not part of the query
@jlcornejo There is no issue here, reflections considered but not matched do not add any extra overhead to planning and execution
the problem is that I need to consume the data in real time, and Dremio will look for a reflection that I don’t have in the from of query VDS.
@jlcornejo Dremio will look for a reflection and if no match will go back to the source
ok,
I need to leave reflection on to consume the story in a VDS.
And another VDS, to have a limited time, in real time, but I can’t do this because it always gets the data from the REFLECTION.
I will see another option to consume data in real time.
thanks!