Hello,
we upgraded Dremio to v22.1.1 community Edition and observed that reflections are not working on a dataset that is using the FLATTEN() function.
Could you tell if it is a known issue and available solutions ?
Without FLATTEN():
With FLATTEN():
@allCag flatten(column) should accelerate, the issue you are facing is different, if you see your reflection status is not complete, can you please click on the history icon to the right of the foot print?
Do you see 2 jobs?
Can you please send us both the profiles?
Thanks
Bali
@balaji.ramaswamy we can see 2 jobs like this using the flatten():
Do you have a secured channel I can send the profiles to ?
@allCag You can email it to balaji@dremio.com
Thanks @balaji.ramaswamy , just sent you both job profiles.
@allCag I see both the reflection jobs are complete, do you see the LOAD MATERIALIZATION for both the REFRESH jobs?
Search for 166b2462-38f2-44be-b4d9-1dc8c4944e52 and 126273ab-d83e-409f-9c4d-147ef3e37325?
Hi @balaji.ramaswamy
yes, we can see it for both:
@allCag The profiles show job is completed, when you say reflections are not working when you use FLATTEN, do you mean the query is not accelerated? Can you please send me the profile of the query that did not use the reflection?
Hi @balaji.ramaswamy just sent on your email detailed examples with job profils with and without the flatten.
@allCag Somehow Dremio does not even list the Flatten reflection. Are you able to send me the json download of sys.reflections?
@balaji.ramaswamy I have a similar issue. support ticket 117449(internal JIRA DX-51303). I can not create reflection on flatten field. The LOAD MATERIALIZATION job is completed on UI, but in server.log you can find some errors.
Thanks @Switch I will follow up internally
Hi @balaji.ramaswamy,
could you give an update on this topic ? Is a fix implemented in Dremio versions above v22 ?
