Configuring NAS source mount in Kubernetes deployment

1

Hi, we’re trialling deploying Dremio on Kubernetes in GCP. Because there isn’t support for GCP cloud buckets yet we’re exploring using NAS to provide a pathway between managed file transfer and Dremio.
Does anyone have an example of a config they’ve used to successfully deploy to Kubernetes with a configured mount point for a NAS to use for making source files available?

Similarly I’ve not been successful configuring a NAS for distributed storage though am most interested in being able to access shared files for source.
Any help appreciated. Thanks for all the work on the container and helm samples.
Cheers
Mike

2

Storage in Kubernetes is still evolving and there are some options out there for shared storage across the nodes. From a Dremio perspective, it is a filesystem that it can access on the containers.

As for NAS, you can mount the NAS source as nfs share on all the containers. If your NAS has nfs server running on nfshost and the path to your directory is /mypath, you can mount the nfs share as:
(On each of the master, coordinator and executor statefulsets):

  1. Define volume like:
   - name: nfs-vol
     nfs:
       server: nfshost
       path: /mypath
  1. Mount the volume:
    - name: nfs-vol
      mountPath: /var/nfs

You can then use /var/nfs (or sub-directories under it) as a NAS data source in Dremio.

3

Thanks for the help, not too sure what I was doing wrong, your guidance was perfect, appreciated! All working now

1 Like
4

Hi @MikeRS,

Did you ever managed to configure disributed storage with NAS? If so could you provide yaml code?

How about authorization on the NAS?