Dremio with docker swarm and reflections status

We are using Dremio with docker swarm as the orchestration tool, and also using anNFS share. Every time we cycle the stack, all reflections gets into an INVALID status. I have two questions:

  1. Are they suppose to self heal on the next refresh cycle?
  2. How can we avoid this from happening?

Thanks

@martinocando

How many executors do you have? Is this NFS share mounted on all executors? Can you please provide us the dremio.conf file?

Thanks
Bali

We have 1 master and 4 executors. And yes, the NFS share is mounted on all executors.

This is dremio.conf from the master node:

#########################

paths: {
local: ${DREMIO_HOME}"/data"

dist: “pdfs://”${paths.local}"/pdfs"
}
services: {
coordinator.enabled: true,
coordinator.master.embedded-zookeeper.enabled: false,
executor.enabled: false
}
zookeeper: “zk_zoo1:2181,zk_zoo2:2181,zk_zoo3:2181”
registration.publish-host:“172.25.24.213”

And the dremio.conf of one of the executors:

###############################

paths: {
local: ${DREMIO_HOME}"/data"

dist: “pdfs://”${paths.local}"/pdfs"
}

services: {
coordinator.enabled: false
}

zookeeper: “zk_zoo1:2181,zk_zoo2:2181,zk_zoo3:2181”
registration.publish-host:"${LOCAL_CONTAINER_IP}"

@martinocando

The dremio.conf setting below is configured as a PDFS, which means the reflection files are distributed across the executors and not like a central store,

dist: “pdfs://”${paths.local}"/pdfs"

please fix it using the below syntax

dist: “file://”${paths.local}"/pdfs"

Thanks for the information. I’ll test it and report back.