Hello, currently we are using Minio as our distributed storage and it works fine for dremio, the config we put in dremio.conf
paths: {
local: ${DREMIO_HOME}"/data"
dist: "dremioS3:///dremio-storage-test/data/"
}
We deploy dremio in K8s (1 master, 3 executors), so the current spill folder can be found inside each executor like:dremio@dremio-executor-0:/opt/dremio/data/spill
Is that possibe we specify the spill directory with Minio path inside dremio.conf ? something like:
paths: {
...
spill: "dremioS3:///dremio-storage-test/spill/"
}
Thanks for help !