How are local uploaded files managed in dremio?

1

Dremio supports uploading files in various formats locally, such as csv, json, etc. How are these files managed in dremio?

2

@fuliu These files are stored in the dist:/// path specified in dremio.conf or your values.yaml

3

Do these files support storage on minio?

4

@fuliu Yes MinIO is supported

7

I want to test dist.path support for minio in idea, here is my vm configuration

-Dpaths.local=/Users/fuliu/dev/dremioData/data

-Dpaths.dist=“dremioS3:///dremio”

-Ddremio.log.path=/Users/fuliu/dev/dremioData/log

-Dlogback.configurationFile=/Users/fuliu/dev/dremioData/conf/logback.xml

-Dservices.coordinator.auto-upgrade=true

-Ddremio.telemetry.configfile=/Users/fuliu/dev/drem ioData/conf/dremio-telemetry.yaml

-Ddremio_allowTestApis=true

-Ddebug.allowNewerKVStore=true

-Dcom.sun.management.jmxremote.port=1099

-Dcom.sun.management.jmxremote.authenticate=false

-Dcom.sun.management.jmxremote.ssl=false

-Dio.netty.tryReflectionSetAccessible=true

-Dservices.coordinator.command-pool.size=20

-Ddremd.mode=“local"

Is this correct? Where should core-site.xml be placed。

Such a configuration cannot be started successfully

8

@fuliu To configure Minio as a source follow instructions in below doc

To Configure MinIO as distributed storage follow below