We are looking at using Dremio on top of our existing Elasticsearch cluster.We had the following queries around ES-Dremio integration
1)Query Interception
Scenario - Support different authorization models
Can we intercept ES queries before they are sent to the database and modify the query or index against which the query is fired
Use cases
a)The same query for two different clients (please note that we are a multi tenant system) must go to two separate ES indexes .
b)We want to add a filter to every ES query to support our authiorization constructs (currently based on ES terms based look up in a different index - https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html#_terms_lookup_twitter_example )
2)Joins and Concurrency
Scenario - Join b/w data in ES and Mysql
I am assuming Dremio must be pushing queries to ES and Mysql ,loading the data in memory or in a DB and performing the joins their . What is the data isolation provided in this temporary store .Is it possible that two users doing exactly the same join operation can see each other’s data ?
3)Special Operations
We want to run some special operations via user interface for eg group by immediate reportees - which gets a user’s immediate reportees from mysql and runs a group by user query on ES including only those users
4)ES features supported
We heavily use ES nested fields and parent child relations .I think they are not supported in Dremio currently ? Are these support in enterprise version