Installation issue CLuster mode

#1

getting the following error & No log files generated

● dremio.service - Dremio Daemon Server
Loaded: loaded (/etc/systemd/system/dremio.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2018-08-29 07:21:03 EDT; 4s ago
Docs: https://docs.dremio.com
Process: 19620 ExecStop=/opt/dremio/bin/dremio stop (code=exited, status=1/FAILURE)
Process: 19605 ExecStart=/opt/dremio/bin/dremio start-fg (code=exited, status=1/FAILURE)
Main PID: 19605 (code=exited, status=1/FAILURE)

Aug 29 05:48:28 hostname systemd[1]: dremio.service failed.
Aug 29 05:48:28 hostname systemd[1]: Started Dremio Daemon Server.
Aug 29 05:48:28 hostname systemd[1]: Starting Dremio Daemon Server…
Aug 29 05:48:28 hostname dremio[10375]: starting dremio, logging to /opt/dremio/log/server.out
Aug 29 05:48:38 hostname systemd[1]: dremio.service: main process exited, code=exited, status=1/FAILURE
Aug 29 05:48:39 hostname systemd[1]: Unit dremio.service entered failed state.
Aug 29 05:48:39 hostname systemd[1]: dremio.service failed.

I am trying to install on a two node Hadoop using Yarn .How does the other node know this node as master. Do we need to specify the hostname explicitly anywhere?

#2

The nodes know of each other via ZK. If you are deploying in a Hadoop cluster, you should use your external ZK (rather than Dremio’s default internal one) - https://docs.dremio.com/advanced-administration/zookeeper.html#using-an-external-zookeeper

Also, you only need to deploy the master coordinator one time. You can stand up the executors via UI via Yarn.

#3

I have two nodes host1 & host2 which is hadoop cluster and both acts as zookeeper. I used setup instructions for both

Master config:

paths: {

the local path for dremio to store data.

local:"/data/dremio"
dist:“hdfs://namenode/user/dremio”
}

services: {
coordinator.enabled: true,
coordinator.master.enabled:true ,
executor.enabled: false
coordinator.auto-upgrade: false
}
services.coordinator.master.embedded-zookeeper.enabled: false
zookeeper: “host1:2181,host2.com:2181”

Log that I get when i start ~
Exception in thread “main” java.lang.NullPointerException: No Cluster Identity found
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:226)
at com.dremio.dac.daemon.DremioDaemon.checkVersion(DremioDaemon.java:89)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:125)

Executor config:

paths: {

the local path for dremio to store data.

local: “/data/dremio”
dist:“hdfs://namenode/user/dremio”

}

services: {
coordinator.enabled: false,
coordinator.master.enabled: false,
executor.enabled: true
coordinator.auto-upgrade: false
}
services.coordinator.master.embedded-zookeeper.enabled: false
zookeeper: “host1:2181,host2.com:2181”

No logs generated

Any help

#4

Could you try to delete content of /data/dremio directory and restart?

1 Like
#5

Well that did not helo. However adding JAVA_HOME to Dremio_env file helped fixing the error