Recent changes to Dremio 21+ (Distributed Storage Local Storage)

1

Hello Everyone,

We are currently running Dremio 20 on Kubernetes in an environment that does not have access to cloud storage. As far as I understand, distributed storage will no longer work with Local. Does that mean we cannot upgrade to Dremio 21 or 22? If we can upgrade, is it only the reflections that will be affected if we do not have cloud storage?

Thanks

2

@OmarSultan85 Yes, reflections and iceberg metadata for Parquet, ORC and AVRO cannot be used

3

You can use NAS inside your k8s

4

Thanks a lot for the clarification, we are also using JSON files that are hosted on a NAS Storage, that will still work right?

5

Thanks dacopan, do you know what the config in the dremio.conf would be? As far as I understand it only allows for local (no longer supported) s3, google and azure. What would the configuration be for NAS?

6

Assuming you have a NAS volume mounted in /data/nas you can use in dremio config

dist: “file:///data/nas”

7

Also if you can’t use NAS, you can use minio as S3 local distribute storage