Error when reading from MySQL

1

Hi,

I get an error when trying to connect to one of our MySQL databases. It contains 49 tables but the initial metadata scan stops at table number 13 with this error: “IllegalArgumentException: Multiple entries with same key: cell_id_key=CELL_ID_KEY: Int(32, true) and cell_id_key=CELL_ID_KEY: Int(32, true)”

I’ve included the query profile.

Thanks!

/Niclas

aa1ab8ec-7c5b-4803-b913-e65816bed6f6.zip (3.5 KB)

2

Hi @roos.niclas,

Can you please send me the “describe of table ress_1.ress.CM_CELL”?

Thanks,
@balaji.ramaswamy

3

Directly in the database or in Dremio?

4

Hi @roos.niclas

Sorry for the delay in response, just wanted to check with you if you are still having trouble adding the MySQL data source. If yes, can you please update with the latest issue. If still on the same issue kindly send me a describe of the table from MySQL?

Thanks,
@balaji.ramaswamy

5

Actually, I just realized that there are two tables called CM_CELL in the database, one in upper case and one in lower case, could that be causing the error?

Also, attached is the create script from the database. I think this gives more info than I get when I run describe.

Thanks,

Niclas

cm_cell.sql.zip (1.0 KB)

6

Hi @roos.niclas,

Is this a development environment? Can we try renaming one of the CM_CELL to a different unique name and give it a try?

Thanks,
@balaji.ramaswamy

7

I am running into same issue with Elastic engine. The index metadata has two columns with same name but the case is different. APPID and appid.

My Elastic admin is saying that changing Elastic metadata is a huge task and not an option. Was there a fix for this issue?

Thanks
Sujan

elasticError

8

Dremio is case preserving and case insensitive. While Elasticsearch does allow multiple columns with different case (APPID and appid), what we’ve realized with other companies using Dremio is that this was not intentional and actually represented a data quality problem in Elasticsearch, which can also cause problems with other systems like Kibana. Dremio’s current behavior is to raise an error when it sees this situation so that you can address it at the source - is that possible? Do you intentionally have two columns with different cases?

9

I’m getting these errors randomly.
example:
Multiple entries with same key: c_mktsegment=c_mktsegment: Utf8 not null and c_mktsegment=c_mktsegment: Utf8 not null. To index multiple values under a key, use Multimaps.index.

once it shows up it never leaves

10

@nadavharpazi Is this a RDBMS or JSON source?