Wrong content when using a raw reflection

1

Hi,
I have a very annoying issue in Dremio. I’m using community version 1.4.4.

I have created an advanced raw reflection on some columns of a MySQL Table.
The content seems wrong especially on a column which has SMALL INT type in Mysql.
This column is not null on the mysql database,
but when Dremio is using the reflection, a group by on this column to get the count by value tells me that
there are 7 911 631 null occurrences for this column, while the table has around 20 Millions rows.
So the content of this column is at least wrong for more than 1/3.

I’m not sure what informations to provide for analysis.

Thanks

2

Hi @dfleckinger,

A query profile would be a good starting point - instructions can be found here:
https://www.dremio.com/tutorials/share-query-profile/

3

Hi @Sharon,

ok, here are the query profiles of :the acceleration creation, and

4

hi @Sharon, the issue still occurs for me on 3.1.1.
Any idea ?

5

Hi @dfleckinger,

Looking at the offline thread we had back then I see that we were not able to reproduce the issue back then with the additional input you had provided.

Couple of questions/requests:

  1. Have you identified a unique pattern on your end? For instance:
  • Does it occur only with that table/reflection or also with other tables/reflections?
    • if that’s the only table being impacted - do you see the same behavior when grouping on other columns or the problem occurs only with that smallint data type?
  1. Can you please specify MySQL version and provide table DDL?
  2. If not done already, can you delete that raw reflection and recreate it and then attach 3.1.x query profiles:
    a. With reflection enabled
    b. With reflection disabled