Dremio kubernetes cluster installation

1

I have been trying to install Dremio in kubernetes.
I was successful with the helm installation however the issue comes when i need to copy the additional custom connector jars from filesystem into the kubernetes pod.
I use “kubectl cp ./testconnector-1.0.0.jar kuberbetes-executor-0:/opt/dremio” and I get the “operation not permitted” error.

Error:
tar: jconn4.jar: Cannot open: Permission denied
tar: Exiting with failure status due to previous errors
command terminated with exit code 2

Is there any way around this. Iam having root credentials.

Tried this: kubernetes - `kubectl cp` to a pod is failing because of Permission Denied - Stack Overflow

but once it goes to tmp I cannot move it to /opt/dremio as i get the same operation not permitted
Error :
cp: cannot create regular file ‘/opt/dremio/jars/testconnector-1.0.0.jar’: Permission denied

File details:
-rwxrwxrwx 1 dremio dremio 17178 Apr 28 13:23 testconnector-1.0.0.jar

How can I get around this issue

2

@gururajnayak

Some useful links to read

https://docs.dremio.com/deployment/amazon-eks/amazon-eks/
https://docs.dremio.com/deployment/azure-aks/azure-aks/

3

Hi All,

I have managed to solve this issue but the question that i have asked is part of a bigger issue.

  1. For the immediate question the anwser is : edit /dremio-cloud-tools/charts/dremio/templates/dremio-executor.yaml and add the below lines
    …
    …
    …

containers:
- name: dremio-executor
image: {{.Values.image}}:{{.Values.imageTag}}
imagePullPolicy: IfNotPresent
######## Below lines are added by me START ########
securityContext:
runAsUser: 0
######## END ########
…
…
…
…

  1. The bigger question is that the above is not enough. The original problem is one needs to copy the connectors to the kubernetes executor pod and restart the pod. So simply copying is not enough.
  2. Ideally the solution is one has to use initcontainer in dremio-executor-deployment.yaml and wget the jars and then start the container.
  3. when i added the initcontainer iam getting the error which i am still struggling with.

…
…
volumeMounts:
- name: dremio-executor-volume
mountPath: /opt/dremio/data
- name: dremio-executor-volume # newly added
mountPath: /opt/dremio/data # newly added
- name: dremio-config
mountPath: /opt/dremio/conf
…
…

then added below

initContainers:

  • name: wait-for-zk
    …
    …

  • name: download-jars
    image: {{.Values.image}}:{{.Values.imageTag}}
    imagePullPolicy: IfNotPresent
    securityContext:
    runAsUser: 0
    volumeMounts:
    - name: dremio-connector-volume
    mountPath: /opt/dremio/jars
    command: ["/bin/sh","-c"]
    args: [“wget --no-check-certificate https://somenexusrepo.com/testconnector-1.0.0.jar”]

then added below in same document. Note the sections

volumeClaimTemplates
…
…

  • metadata:
    name: dremio-connector-volume
    spec:
    accessModes: [ “ReadWriteOnce” ]
    {{- if .Values.storageClass }}
    storageClassName: {{ .Values.storageClass }}
    {{- end }}
    resources:
    requests:
    storage: {{.Values.executor.volumeSize}}

However after saving and going helm upgrade the container is not starting. anybody has any idea.