HDFS Folder for high availability

Hi all,

in order to have two masters in high availability mode, i’d like to know if HDFS could be a good choice to share local metadata between the two masters.

I’ve seen in documentation NFS, SMB/CIFS or similar…

thanks in advance.

Unfortunately, no: at that time, Dremio doesn’t support HDFS as a store for the local metadata (it only supports a local mountpoint/volume so that’s why advise for NFS or similar).

Is it in the pipe for next versions?

The nature of Dremio’s metadata management relies on a DBMS and is therefore not really suitable for HDFS.

Is there something about the current dependencies that makes deployments difficult? We’d love to hear your thoughts on this topic.

Hi Kelly, I’m deploying the Dremio to local stateless cloud(no mounted persistence volumes), and every redeployment the catalog gets ruined.