Unable to deploy dremio on kubernetes,

1

I am trying to deploy dremio on my kubernetes cluster.
I have the following values.yaml

# The image used to build the Dremio cluster. It is recommended to update the
# version tag to the version that you are using. This will ensure that all
# the pods are using the same version of the software.
image: dremio/dremio-oss:3.2.4
# Check out Dremio documentation for memory and cpu requirements for
# the coordinators and the executors.
# The value of memory should be in MB. CPU is in no of cores.
coordinator:
  memory: 4096
  cpu: 2
  # This count is for slave coordinators only.
  # The chart will always create one master coordinator - you are
  # not required to have more than one master coordinator.
  count: 0
  web:
    port: 9047
  client:
    port: 31010
  volumeSize: 100Gi
executor:
  memory: 4096
  cpu: 2
  count: 1
  volumeSize: 100Gi
zookeeper:
  memory: 1024
  cpu: 0.5
  count: 1
  volumeSize: 10Gi

# To create a TLS secret, use the following command:
# kubectl create secret tls ${TLS_SECRET_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}
tls:
  ui:
    # To enable TLS for the web UI, set the enabled flag to true and provide
    # the appropriate Kubernetes TLS secret.
    enabled: false
    secret: dremio-tls-secret-ui
  client:
    # To enable TLS for the client endpoints, set the enabled flag to true and provide
    # the appropriate Kubernetes TLS secret. Client endpoint encryption is available only on
    # Dremio Enterprise Edition and should not be enabled otherwise.
    enabled: false
    secret: dremio-tls-secret-client

# If your Kubernetes cluster does not support LoadBalancer,
# comment out the line below for the helm chart to succeed or add
# the correct serviceType for your cluster.
# serviceType: LoadBalancer

# If the loadBalancer supports sessionAffinity and
# you have one or more coordinators, enable it
#sessionAffinity: ClientIP

# Enable the following flag if you wish to route traffic through a shared VPC
# for the LoadBalancer's external IP.
# The chart is setup for internal IP support for AKS, EKS, GKE.
# For more information, see https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
#internalLoadBalancer: true

# To use custom storage class, uncomment and provide the value here.
# Otherwise the default storage class configured for your K8S cluster is used.
storageClass: ebs-gp2-alt

# For private and protected docker image repository, you should store
# the credentials in a kubernetes secret and provide the secret name here.
# For more information, see https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
#imagePullSecrets: secretname

# Target pods to nodes based on labels set on the nodes.
# For more information, see https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
#nodeSelector:
#  key: value

# Control where uploaded files are stored.
# See https://docs.dremio.com/deployment/distributed-storage.html for more information
dremioVersion: "3.2.0" # Dremio Version 3.2.0 or greater
distStorage:
  # Valid values are local, aws, azure or azureStorage. aws and azure choice requires additional configuration data.
  type: "aws"
  aws: #S3 - used for only uploads
    bucketName: "dremio"
    path: "/dremio"
    accessKey: "---"
    secret: "---/---"

Get Pods

> kubectl get pods -n dremio-02
NAME                READY   STATUS                  RESTARTS   AGE
dremio-executor-0   1/1     Running                 0          11m
dremio-executor-1   1/1     Running                 0          11m
dremio-executor-2   1/1     Running                 0          11m
dremio-master-0     0/1     Init:CrashLoopBackOff   6          9m3s
zk-0                1/1     Running                 0          11m

> kubectl logs dremio-master-0 -c dremio-master-coordinator -n dremio-02
Error from server (BadRequest): container "dremio-master-coordinator" in pod "dremio-master-0" is waiting to start: PodInitializing

I have tried debugging nearly everything, can anyone help.

2

Rohithzr,

I just ran into this yesterday and managed to solve it. I am using 3.2.4 as well. In my dremio.conf file I originally had:
dist: "dremioS3://s3bucket/..."

Once I changed dremioS3 to s3a it worked. So I ended up with:
dist: "s3a://s3bucket/..."

Looking at the documentation (https://docs.dremio.com/deployment/dist-store-config.html) the example has dremioS3:///s3bucket (note the 3 slashes). I wonder if this could have been the problem too. I will test that when I get a chance.

1 Like
3

@mlilius

Thanks a lot mate, s3a:// is the solution. I could not get it working with triple slash.

4

While working with s3 distributed storage, dremioS3:///path (capital S and three slashes - ///) worked fine for me. Also did s3a://path. I don’t know if they bear any significant differences.

(The dremio.conf file also has to be identical in every node, or else zookeper won’t work properly.)