I have a very annoying issue in Dremio. I’m using community version 1.4.4.
I have created an advanced raw reflection on some columns of a MySQL Table.
The content seems wrong especially on a column which has SMALL INT type in Mysql.
This column is not null on the mysql database,
but when Dremio is using the reflection, a group by on this column to get the count by value tells me that
there are 7 911 631 null occurrences for this column, while the table has around 20 Millions rows.
So the content of this column is at least wrong for more than 1/3.
I’m not sure what informations to provide for analysis.
A query profile would be a good starting point - instructions can be found here:
ok, here are the query profiles of :the acceleration creation, and
hi @Sharon, the issue still occurs for me on 3.1.1.
Any idea ?
Looking at the offline thread we had back then I see that we were not able to reproduce the issue back then with the additional input you had provided.
Couple of questions/requests:
- Have you identified a unique pattern on your end? For instance:
- Does it occur only with that table/reflection or also with other tables/reflections?
- if that’s the only table being impacted - do you see the same behavior when grouping on other columns or the problem occurs only with that smallint data type?
- Can you please specify MySQL version and provide table DDL?
- If not done already, can you delete that raw reflection and recreate it and then attach 3.1.x query profiles:
a. With reflection enabled
b. With reflection disabled