Having some trouble setting up the external zookeeper quorum for HA with Dremio.
I downloaded the vanilla zk tarball from Apache site. The only change I made was to the config file.
zoo.cfg:
the maximum number of client connections.
server1=<zk1.server>:2888:3888
server2=<zk2.server>:2888:3888
server3=<zk3.server>:2888:3888
dremio.conf on each of 2 coordinators:
paths: {
the local path for dremio to store data.
local: ${DREMIO_HOME}"/data"
the distributed path Dremio data including job results, downloads, uploads, etc
#dist: “pdfs://”${paths.local}"/pdfs"
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true
coordinator.master.embedded-zookeeper.enabled: false,
coordinator.master.embedded-zookeeper.port: 2181,
coordinator.web.enabled: true,
coordinator.web.port: 9047,
coordinator.web.auth.type: “ldap”,
coordinator.web.auth.ldap_config: “ad.json”,
coordinator.client-endpoint.port: 31010,
executor.enabled: false,
fabric.port: 45678
}
zookeeper: “<zk1.server>:2181,<zk2.server>:2181,<zk3.server>:2181”
I can access the Dremio Coordinator just fine but no executors connect.
dremio.conf on each executor:
zookeeper: “<zk1.server>:2181,<zk2.server>:2181,<zk3.server>:2181”
paths: {
the local path for dremio to store data.
local: ${DREMIO_HOME}"/data"
the distributed path Dremio data including job results, downloads, uploads, etc
#dist: “pdfs://”${paths.local}"/pdfs"
}
services: {
coordinator.enabled: false,
coordinator.master.enabled: false,
executor.enabled: true
}
Executor logs just say:
2018-05-06 14:23:14,990 [main] INFO c.d.d.s.exec.MasterStatusListener - Waiting for master