HDFS Folder for high availability

1

Hi all,

in order to have two masters in high availability mode, i’d like to know if HDFS could be a good choice to share local metadata between the two masters.

I’ve seen in documentation NFS, SMB/CIFS or similar…

thanks in advance.

2

Unfortunately, no: at that time, Dremio doesn’t support HDFS as a store for the local metadata (it only supports a local mountpoint/volume so that’s why advise for NFS or similar).

3

thanks.
Is it in the pipe for next versions?

4

The nature of Dremio’s metadata management relies on a DBMS and is therefore not really suitable for HDFS.

Is there something about the current dependencies that makes deployments difficult? We’d love to hear your thoughts on this topic.

5

Hi Kelly, I’m deploying the Dremio to local stateless cloud(no mounted persistence volumes), and every redeployment the catalog gets ruined.