Specify a custom spill directory in Minio

Hello, currently we are using Minio as our distributed storage and it works fine for dremio, the config we put in dremio.conf

paths: {
  local: ${DREMIO_HOME}"/data"
  dist: "dremioS3:///dremio-storage-test/data/"
}

We deploy dremio in K8s (1 master, 3 executors), so the current spill folder can be found inside each executor like:dremio@dremio-executor-0:/opt/dremio/data/spill

Is that possibe we specify the spill directory with Minio path inside dremio.conf ? something like:

paths: {
  ...
  spill: "dremioS3:///dremio-storage-test/spill/"
}

Thanks for help !

1 Like

@alpha94511 Spilling till Minio is not recommended as you may run into performance issues

Thanks for response !