Hi,
I have a question regarding pushdowns for queries on elasticsearch analyzed text fields. I was having an issue using a WHERE clause on a text field where Dremio was not pushing the filter down to elasticsearch and thus the query was taking a long time to execute(since it was just doing a select all and bringing all the data back into Dremio).
After reading the documentation, I selected the option “Use index on analyzed fields and normalized fields” for the elasticsearch connection. The description for this is:
“Use this option to enable pushdown filters and aggregations on analyzed text fields and normalized keyword fields”
This then resolved the issue in that the WHERE filter was pushed down to the elasticsearch query, eg using a match query.
However, I have noticed that it will still not push down a GROUP BY clause for text fields. eg if I do a GROUP BY on a numeric field it will be pushed down as a terms aggregation but on the text field there is still no push down therefore the query takes a long time again.
According to the documentation, the setting I checked should push down for both filters and aggregations on normalised text fields however this does not appear to be happening. It works for filters but not for aggregations.
Is there some other configuration I am missing or is this actually not supported in Dremio?
Thanks
Adam