Issue in dist. pdfs with multiple dremio setup

1

Hi,

We are facing issues while choosing pdfs to distributed file system, in our case its a file system shared across both the nodes. We pointed to shared file system in both the dremio.conf configuration file.

In Our current system we configured two servers (1 with coordinator and executor and 2nd is only executor),also additional config on master node for the zookeeper.

When ever we point to shared mount pt. for pdfs, dremio doesnt show us the nodes. So we have revert back the changes ( # out the dist parameter)

Regards
Ankit S

2

You cannot use PDFS with a shared filesystem as each node expects to fully own the content of the directory to store data. Simply points PDFS to a non-shared directory on each node and you should be able to see nodes again.

3

As @laurent pointed out you can’t have PDFS pointing to the same mount point, but you can switch from using pdfs://…
to any other distributed file system - e.g. hdfs if you decide to do so