Is there any possibility where we can isolate the dremio coordinator related services running on kubernetes(spinned up using Helm chart).
When Dremio is spinned up using Helm charts, there are 3 Kubernetes Services spinned up -
dremio-client - Inside this service, 3 ports are exposed: 31010(ODBC), 9047(WebUI), 32010(ArrowFlight).
dremio-cluster-pod
zk
Query: Is it possible to have dremio-client service splitted to 3 separate services - one for ODBC(Port 31010), Web UI(port 9047) and one service for ArrowFlight(Port 32010)?
Hey @ramprasd89 - you can do this by modifying this template here: dremio-cloud-tools/dremio-service-client.yaml at master · dremio/dremio-cloud-tools · GitHub
But just to understand better - may I know the use-case why you want to do this?
Hi @lenoyjacob - Thanks for the quick response!
In the template here - would you recommend us to have 3 different services by only altering the spec.ports.port, with each service having only their respective port/targetport/name parameters?
As requested, please find more details on the use case below:
We have deployed Dremio on an AKS cluster and have exposed the UI behind an Azure App Gateway, with WAF features enabled. The UI(exposed as a service - dremio-client - on port 9047) is accessible behind the App Gateway without any issues, however when we tried to establish ArrowFlight connection(which is again exposed on same service - dremio-client - on port 32010), there were issues with the App GW not being able to realize the HTTP2 protocol, arrowflight uses. As a result, we were forced to split the “dremio-client” into 2 - one for the UI(port 9047 and routed through AppGW) and the other for arrowflight connection(port 32010 and routed through layer 4 LoadBalancer). Hence the need for splitting the services.
More details could be found in the ticket here - Accessing Dremio running behind App Gateway from Superset