we upgraded Dremio to v22.1.1 community Edition and observed that reflections are not working on a dataset that is using the FLATTEN() function.
Could you tell if it is a known issue and available solutions ?
@allCag flatten(column) should accelerate, the issue you are facing is different, if you see your reflection status is not complete, can you please click on the history icon to the right of the foot print?
Do you see 2 jobs?
- REFRESH REFLECTION
- LOAD MATERIALIZATION
Can you please send us both the profiles?
@balaji.ramaswamy we can see 2 jobs like this using the flatten():
Do you have a secured channel I can send the profiles to ?
Thanks @balaji.ramaswamy , just sent you both job profiles.
@allCag I see both the reflection jobs are complete, do you see the LOAD MATERIALIZATION for both the REFRESH jobs?
yes, we can see it for both:
@allCag The profiles show job is completed, when you say reflections are not working when you use FLATTEN, do you mean the query is not accelerated? Can you please send me the profile of the query that did not use the reflection?
Hi @balaji.ramaswamy just sent on your email detailed examples with job profils with and without the flatten.
@allCag Somehow Dremio does not even list the Flatten reflection. Are you able to send me the json download of
@balaji.ramaswamy I have a similar issue. support ticket 117449(internal JIRA DX-51303). I can not create reflection on flatten field. The LOAD MATERIALIZATION job is completed on UI, but in server.log you can find some errors.
Thanks @Switch I will follow up internally
could you give an update on this topic ? Is a fix implemented in Dremio versions above v22 ?
@allCag @Switch We are unable to reproduce this in 24.1, can you please try?