Hey there,
You cannot deploy executors from the UI. This is a deployment activity that is done outside of Dremio. See here for details: https://docs.dremio.com/deployment/
The default install is a single node install, where that node is set to the master, coordinator, and executor, so you wouldn’t typically see this error unless you’d changed the configuration files already. So, the fact that you’re seeing this suggests you’ve already the dremio.conf file and unset the master as an executor.
Behind the scenes, Dremio uses zookeeper to co-ordinate the nodes. By default it uses a built-in zookeeper for ease of use. See https://docs.dremio.com/deployment/dremio-config.html for details on what settings to set each executor so they can see the master.
If this still isn’t working, feel free to share the the config files here.
Christy
I used mapr, do I need to install, configured and start the dremio at every host in the mapr cluster?
Sorry, when you said you used MapR, do you mean you used Yarn provisioning under MapR?
If so, this page should help: https://docs.dremio.com/deployment/yarn-deployment.html
If you aren’t using Yarn provisioning, then you shouldn’t need to install Dremio on each node in your MapR cluster. But you will need to manually start the executors. Note, in this scenario, ensure you follow the following start-up and shut-down order:
-
Start Coordinators
-
Start Executors
-
Stop Executors
-
Stop Coordinators
It seems your (master) coordinator is started, so ensure your executors are started too.
Could you share the server.log and server.out from your master node? Hopefully that will give a bit more detail on the error.
Caused by: twill.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Path must not end with / character
106859 at twill.com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106860 at twill.com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106861 at org.apache.twill.zookeeper.ForwardingZKClientService.startAndWait(ForwardingZKClientService.java:51) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106862 at org.apache.twill.yarn.YarnTwillRunnerService.startUp(YarnTwillRunnerService.java:350) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106863 at org.apache.twill.yarn.YarnTwillRunnerService.access$000(YarnTwillRunnerService.java:112) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106864 at org.apache.twill.yarn.YarnTwillRunnerService$3.startUp(YarnTwillRunnerService.java:171) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106865 at twill.com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106866 … 1 common frames omitted
106867 Caused by: java.lang.IllegalArgumentException: Path must not end with / character
106868 at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:58) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106869 at org.apache.zookeeper.client.ConnectStringParser.(ConnectStringParser.java:57) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106870 at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:443) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106871 at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:380) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106872 at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.createZooKeeper(DefaultZKClientService.java:537) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-ma
106873 at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.doStart(DefaultZKClientService.java:447) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106874 at twill.com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106875 at org.apache.twill.internal.zookeeper.DefaultZKClientService.start(DefaultZKClientService.java:306) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106876 at org.apache.twill.zookeeper.ForwardingZKClientService.start(ForwardingZKClientService.java:46) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106877 … 6 common frames omitted
Just as a quick thought, have you set this in your dremio.conf files:
services.coordinator.master.embedded-zookeeper.enabled: false
This ensures Dremio uses the external zookeeper rather than the internal one. You’ll need to set this (and the zookeeper settings) for all nodes
yes, I already set this in dremio config file. I don’t understand you said that I need to set it for all node. I have only one config file which is on the master node. I don’t know where I can deploy other nodes. I only have one master node right now, I thought if I can add the Yarn provisioning, I will have some executors. Is that right?
Sorry, you’re right. Long day, I forgot you’d mentioned you deploy using Yarn.
Could you show content of your dremio.conf and dremio-env?
paths: {
the local path for dremio to store data.
local: “/var/lib/dremio”
the distributed path Dremio data including job results, downloads, uploads, etc
dist: “maprfs://data/chenlini/pdfs”
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: false
}
zookeeper: “host1:5181,host2:5181,host3:5181/dremio2/”
I did not add anything to dremio-env
Please try
zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2”
I tried, if I use “zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2””
the log is:
connect: host1:5181 zkroot: dremio2,host2:5181/dremio2,host3:5181 clusterId: dremio
the clusterId is still dremio. Actually there is already a dremio cluster named “dremio” in our system, I need create another dremio cluster. If I config like that, I will connect with the old cluster.
I forgot to mention - you said you added - services.coordinator.master.embedded-zookeeper.enabled: false
However, you shared your dremio.conf and it is not listed there. Try
zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2”
services.coordinator.master.embedded-zookeeper.enabled: false
Sorry, I just check it and find I already added it. It still shows the same log.
Can you share your full complete dremio.conf please? Also FYI any change to our conf will require a service restart.
paths: {
local: “/var/lib/dremio”
dist: “maprfs://data/chenlini/pdfs”
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: false
}
services.coordinator.master.embedded-zookeeper.enabled: false
zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2”
this is the full complete dremio.conf, I did not add any thing else.
now the error info is “no cluster identity found”.
Exception in thread “main” java.lang.NullPointerException: No Cluster Ident
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java
at com.dremio.dac.daemon.DremioDaemon.checkVersion(DremioDaemon.java:89
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:125)