Helm installation failed - what is wrong with my StatefulSet in version "v1" cannot be handled as a StatefulSet: quantities must match the regular expression

1

Hello,

I try to install Dremio on a Azure Kubernetes cluster through the cloud shell. I customised values.yaml file to fit my cluster size.
But when I try to install dremio with helm I got this error:

$ helm install dremiov2 dremio-cloud-tools/charts/dremio_v2 -f custom_values.yaml

Error: INSTALLATION FAILED: failed to create resource: StatefulSet in version "v1" cannot be handled as a StatefulSet: quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$'

It doesn’t really match other errors on google. Does anyone understand what is going on ?

I also paste my custom values.yaml file so you can try it at home.

image: dremio/dremio-oss
imageTag: latest
annotations: {}
podAnnotations: {}
labels: {}
podLabels: {}
nodeSelector: {}
tolerations: []
coordinator:
  cpu: 2
  memory: 1047552
  count: 0
  volumeSize: 1023Gi
  web:
    port: 9047
    tls:
      enabled: false
      secret: dremio-tls-secret-ui
  client:
    port: 31010
    tls:
      enabled: false
      secret: dremio-tls-secret-client
  flight:
    port: 32010
    tls:
      enabled: false
      secret: dremio-tls-secret-flight
executor:
  cpu: 2
  memory: 1047552
  engines: ["default"]
  count: 1
  volumeSize: 1023Gi
  cloudCache:
    enabled: true
    volumes:
    - size: 512Gi
zookeeper:
  image: zookeeper
  imageTag: 3.8.0
  cpu: 1
  memory: 2048
  count: 1
  volumeSize: 10Gi
distStorage:
  type: "azureStorage"
  gcp:
    bucketName: "GCS Bucket Name"
    path: "/"
    authentication: "auto"
  aws:
    bucketName: "AWS Bucket Name"
    path: "/"
    authentication: "metadata"
  azure:
    datalakeStoreName: "Azure DataLake Store Name"
    path: "/"
    credentials:
      applicationId: "Azure Application ID"
      secret: "Azure Application Secret"
      oauth2Endpoint: "Azure OAuth2 Endpoint"
  azureStorage:
    accountName: "dlsdremiodev"
    filesystem: "sample"
    path: "/"
    credentials:
      accessKey: "my access key"
extraVolumes: []
extraVolumeMounts: []
service:
  type: LoadBalancer
2

Hello Nicolas-Malgat,

I ran your values through a dry run and it is failing because you are asking for a terabyte of ram for the coordinator and executors and which converts into this number in the helm chart

helm install test . -f ~/Documents/code/values.yaml --dry-run | grep memory
W1209 07:47:34.274173   29875 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    # Max total memory size (in MB) for the Dremio process
    # of max memory and the one that is set.
    # Max heap memory size (in MB) for the Dremio process
    # Max direct memory size (in MB) for the Dremio process
    # Max permanent generation memory size (in MB) for the Dremio process
            memory: 1.047552e+06M
            memory: 1.047552e+06M
            memory: 1.047552e+06M
            memory: 2048M

if you allocate say 947552 instead it works

helm install test . -f ~/Documents/code/values.yaml --dry-run | grep memory
W1209 07:48:17.512170   30319 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    # Max total memory size (in MB) for the Dremio process
    # of max memory and the one that is set.
    # Max heap memory size (in MB) for the Dremio process
    # Max direct memory size (in MB) for the Dremio process
    # Max permanent generation memory size (in MB) for the Dremio process
            memory: 947552M
            memory: 947552M
            memory: 947552M
            memory: 2048M
3

Hey thank you for your answer Ryan !

It looks like my file was incorrectly written, I managed to correct it.
I saw that memory problem in kubernetes and modified my values.yaml accordingly.
Now I am stuck on another kubenetes problem, my pods can’t connect their storage disk by themself. But this is another story …