Is it possible to run a dremio cluster with yarn, without using hdfs setup. S3 (minio) will be used for storage layer. If yes, how would we configure it, as the documentation specifies that a namenode address must be given?
@nishant.env Wondering how Yarn allows that for aother applications, as containers are created using Resource Manager
Dear @balaji.ramaswamy, you’re right. However, I am talking about namenode here (for hdfs), not resource manager. I am planning to use s3 for storage, instead of hdfs.
@nishant.env I assume your compute is on Hadoop using Yarn and your data sits on Minio S3, so yes in the Dremio UI give the Hadoop NN and RM and in dremio.conf use dist:///s3