ElasticSearch driver doesn’t support many features.
I believe those features are presented as unexplained errors.
It’s not really documented, and there are no advanced options for this, but that is what I found out so far -
- No support for ip type - causing a NumberFormatError exception
- No support for nested, array types - causing all king of unexplained and undetailed errors, about the structure of the index.
Most trials so far to deal with the conversion of elasticsearch structure use flattening for the json hierarchy which is also not supported in Dremio’s driver.
(Exposing format of field1.nested_field2.nested_field3)
Also, you’ll find out soon enough, that array flattening is needed.
I can show references to such implementations if needed.
Without that most of the indexes out there will not be processed right within Dremio.
Also control of some parameters like scrolling timewait, etc, should be there is order to be able to tune performance when needed.