Hey Guys
I’m getting the following error message:
Multiple entries with same key: posku-str=poSku-str: Utf8 and posku-str=poSKU-str: Utf8. To index multiple values under a key, use Multimaps.index.
The data source is an elasticsearch index which has the following fields in different cases:
- poSku-str
- poSKU-str
Short of deleting the offending source data from the elasticsearch environment, is there any way I can get dremio to overlook that issue and run queries as normal against that data source?
Here are the things I’ve tried so far:
- looking for the concept of Multimaps.index in dremio - it doesn’t look like it’s a configurable feature in dremio
- alter table on INFORMATION_SCHEMA.COLUMNS to change the offending column name to something else - dremio prevents direct modifications to this table
- delete from INFORMATION_SCHEMA.COLUMNS to remove the offending column record - dremio prevents direct modifications to this table
- refreshing the dremio metadata with a reduced scroll size to avoid picking up the offending elasticsearch document - unlike sql data sources, it looks like the dremio metadata is derived the elasticsearch index mappings and not from the elasticsearch document itself
- modifying the elasticsearch index mappings - elasticsearch does not allow modifications to index mappings, even if the existing elsaticsearch index was deleted and an index mapping was manually created, upon first receiving offending data, the offending mapping would get automatically added to the index mapping anyway
Is there a special mode I can put dremio in that allow me to change the data in the INFORMATION_SCHEMA.COLUMNS table?
Thanks