Cannot install dremio in kubernetes

1

Hello
I am trying to install dremio in a kubernetes (microk8s) node.
I am following the steps in

and

But after running microk8s helm3 install dremio dremio_v2 this is the error I get:
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: resource mapping not found for name: “zk-pdb” namespace: “” from “”: no matches for kind “PodDisruptionBudget” in version “policy/v1beta1”
ensure CRDs are installed first
I am quite new with Kubernetes and I don´t know what else I can do. I also checked zookeper.yaml file but it seems OK to me.
can you please help?
Thanks in advance

2

I was able to reproduce this using microk8s and helm3. The fix for me was updating the zookeeper.yaml file to use “apiVersion: policy/v1”.

… charts/dremio_v2/templates/zookeeper.yaml

Change this line:
apiVersion: policy/v1beta1

to this:
apiVersion: policy/v1

This appears to be due to the newer Kubernetes no longer supporting the older policy/v1beta1. Let me know if this works!

3

Hello tgavin
Thanks for the replay.
I modified zookeeper.yaml file and it apparently worked as it responded back with no errors.
However, when I run
microk8s kubectl get services
The ExternalIP is always none and I need to launch the browser from another computer.
I did the kubectl port-forward dremio-master-0 9047:9047 31010:31010 and the message is:
error: unable to forward port because pod is not running. Current status=Pending

How can I make it work?

Thanks
Oskar

4

I finally made it work!! I had an issue in my microk8s configuration.
I created a docker image derived from dremio-oss:20.1.0 (GIS plugins) and import that image into microk8s. The I modified de values.yaml file so that it gets that image. I fails though, the error says that it is pointing to the latest image in dockerhub.
How can I pull the image from the local registry?
Many thanks
Oskar

5

Hi Oscar,

To reference an image from a different possibly private registry, you need only prefix with the right hostname:

image: my_private.registry.com/dremio/dremio-oss
imageTag: "21.2"

Where “my_private.registry.com” is the hostname to the private registry.

If it’s not a private registry, but simply on Docker Hub under a different account, you need only change the account prefix and possibly image name and tag assigned. Akin to the following:

image: personal_account/my_image_name
imageTag: "1.0"

Thanks,
–Reid

6

Hi Reid
I pushed a Docker image to DockerHub
https://hub.docker.com/r/txalaparta/dremio/tags
And then modified local.values.xml

image: txalaparta/dremio-gis
imageTag: 1.0

However, when I launch
microk8s helm3 install dremio dremio_v2 -f ./local.values.yaml
It fires the following error:
Error: INSTALLATION FAILED: template: dremio/templates/dremio-master.yaml:22:39: executing "dremio/templates/dremio-master.yaml" .... template: dremio/templates/dremio-master.yaml:301:11: e xecuting "dremio/templates/dremio-master.yaml" at <eq $.Values.imageTag "latest">: error calling eq: incompatible types for comparison
What do you think I´m doing wrong?
Thanks again
Oskar

7

Hello Oskar,

I ran into this once before myself. The fix was to quote the image tag so it’s interpreted as a string and not a number. Once I did that, the chart was able to make the string to string comparison that the template is attempting against the “latest” tag value.

Let me know if you still have troubles.

Thanks,
–Reid

8

Hi Reid
I quoted the imagetag value and it worked for me!!!
I have one last question. As I mentioned, in the docker image I just copied a few plugin files. Is there any other way to copy these files via values.yaml file? I mean, without using a derived docker image.

Many many thanks for your help
Oskar

9

Hello Oskar,

I am happy to hear that solved the issue for you!

Not having full context of what exactly you are adding to the image, it’s hard for me to say with 100% surety. However, the current iteration of the Helm Charts don’t have a lot of flexibility for adding additional Volumes and Volume Mounts into the deployment, which is often a path to apply additional things into a Kubernetes based deployment. But with that context, I would anticipate that the only mechanism for customizing what is deployed is through a customized Docker image, much as you have done.

Though it’s not that common to have to do this, I have had to do this kind of effort for my iSCSI driver in my K8s cluster; since the current csi-driver-iscsi doesn’t correctly handle my Ubuntu 22.04 based nodes. Thus, it is not unheard of to need to do that sort of image customization pre-deployment, versus attaching an additional Volume and configuring to reference that newly provided data or library.

It all depends upon how flexible the Helm Chart has been written to afford this level of deployment customization without need to customize the images used by the application.

So, long story short, I think the approach you are using is likely the way you will need to do that, if you are needing to add additional .jar files for instrumentation of whatnot into the equation. If you can provide more details of what you need to add, I might be able to answer more pointedly to your actual use case.

Thanks,
–Reid

10

Hi Reid
This is my Dockerfile where I copy the required jar files

FROM dremio/dremio-oss:20.1.0

COPY esri-geometry-api-2.2.2.jar   /opt/dremio/jars/3rdparty
COPY proj4j-0.1.0.jar  /opt/dremio/jars/3rdparty
COPY dremio-udf-gis-0.2.3.jar  /opt/dremio/jars/3rdparty
COPY sqlite-jdbc-3.36.0.3.jar  /opt/dremio/jars/3rdparty
COPY dremio-sqlite-plugin-20.1.0-202202061055110045-36733c65.jar   /opt/dremio/jars/

Do you think there is away to copy them to the pods without using a new docker image?
Thanks
Oskar

11

I was thinking in copying the files in a persistent store and then somwehow run a copy command in a yaml file. Would that be possible?
Thanks again
Oskar

12

Hello Oskar,

A modified Docker image is probably the simplest solution to what you are trying to accomplish. Otherwise, the way to “copy using YAML file” would be to modify the Helm Chart itself, adding an additional InitContainer which would perform the copy ahead of the Dremio software startup. Bear in mind that this path has more requirements and setup to make work from the get-go.

Since you already have your hands around the custom Docker image, I would go with that route, as it’s easier to handle upstream changes to a Custom Docker image, versus taking upstream changes to the Helm Charts.

With the custom Image approach, you just need to modify your FROM tag to get the new Dremio version into play, and fire off a new build of the Container. However, if you modify the Helm Chart, you have to merge any upstream changes into your local version, when and if such fixes should be posted. Also, you have the complexity of adding an additional InitContainer and setting up this ancillary Persistent Store you mentioned from which you would need to grab these other jar files.

Doing the work ahead of time through a modified image does not impact Dremio start-up time like an additional InitContainer will do. This direction also takes away additional pre-setup work for a new standup of Dremio, for instance, as might happen with a Disaster Recovery operation.

So, I recommend that you continue down the path you are going with a modified Container image. It will generally be the least overall effort to accomplish the desired goal. Once it’s set and working the way you need, you only need to rebuild a new image at those times when you want to take a new version of Dremio into your environment, with no other triggers of the new effort.

Plus, I am not sure how familiar you are with templating Helm Charts, but you seem to know well enough about what you are doing with the Docker image. You might as well keep the focus on that since you seem to be close to the solution you seek. Plus, a late redirection of implementation might extend the overall timeline of completion, which may or may not be a concern for you.

Personally, when it comes to a choice between modifying a Container or a Helm Chart to deploy a solution, I tend to go with “least effort” using a modified image approach. I will only modify the Chart when there is some brittle aspect to it that prevents me from getting the application deployment I need. Perhaps in the case that the Chart and values.yaml are unable to control some specific capability I seek to implement, like cert-manager integration for instance.

Thanks,
–Reid