yes, I used Yarn.
I am already in this step. But I can’t set up mapr with yarn provisioning.Could you share the server.log and server.out from your master node? Hopefully that will give a bit more detail on the error.
Caused by: twill.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Path must not end with / character
106859 at twill.com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106860 at twill.com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106861 at org.apache.twill.zookeeper.ForwardingZKClientService.startAndWait(ForwardingZKClientService.java:51) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106862 at org.apache.twill.yarn.YarnTwillRunnerService.startUp(YarnTwillRunnerService.java:350) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106863 at org.apache.twill.yarn.YarnTwillRunnerService.access$000(YarnTwillRunnerService.java:112) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106864 at org.apache.twill.yarn.YarnTwillRunnerService$3.startUp(YarnTwillRunnerService.java:171) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106865 at twill.com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106866 … 1 common frames omitted
106867 Caused by: java.lang.IllegalArgumentException: Path must not end with / character
106868 at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:58) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106869 at org.apache.zookeeper.client.ConnectStringParser.(ConnectStringParser.java:57) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106870 at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:443) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106871 at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:380) ~[zookeeper-3.4.5-mapr-1604.jar:3.4.5-mapr-1604–1]
106872 at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.createZooKeeper(DefaultZKClientService.java:537) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-ma
106873 at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.doStart(DefaultZKClientService.java:447) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106874 at twill.com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106875 at org.apache.twill.internal.zookeeper.DefaultZKClientService.start(DefaultZKClientService.java:306) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106876 at org.apache.twill.zookeeper.ForwardingZKClientService.start(ForwardingZKClientService.java:46) ~[dremio-twill-shaded-2.0.1-201804132202000759-10b1de0-mapr.jar:2.0.1-201804132202000759-10b1de0-mapr]
106877 … 6 common frames omitted
Just as a quick thought, have you set this in your dremio.conf files:
services.coordinator.master.embedded-zookeeper.enabled: false
This ensures Dremio uses the external zookeeper rather than the internal one. You’ll need to set this (and the zookeeper settings) for all nodes
yes, I already set this in dremio config file. I don’t understand you said that I need to set it for all node. I have only one config file which is on the master node. I don’t know where I can deploy other nodes. I only have one master node right now, I thought if I can add the Yarn provisioning, I will have some executors. Is that right?
Sorry, you’re right. Long day, I forgot you’d mentioned you deploy using Yarn.
Could you show content of your dremio.conf and dremio-env?
paths: {
the local path for dremio to store data.
local: “/var/lib/dremio”
the distributed path Dremio data including job results, downloads, uploads, etc
dist: “maprfs://data/chenlini/pdfs”
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: false
}
zookeeper: “host1:5181,host2:5181,host3:5181/dremio2/”
I did not add anything to dremio-env
Please try
zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2”
I tried, if I use “zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2””
the log is:
connect: host1:5181 zkroot: dremio2,host2:5181/dremio2,host3:5181 clusterId: dremio
the clusterId is still dremio. Actually there is already a dremio cluster named “dremio” in our system, I need create another dremio cluster. If I config like that, I will connect with the old cluster.
I forgot to mention - you said you added - services.coordinator.master.embedded-zookeeper.enabled: false
However, you shared your dremio.conf and it is not listed there. Try
zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2”
services.coordinator.master.embedded-zookeeper.enabled: false
Sorry, I just check it and find I already added it. It still shows the same log.
Can you share your full complete dremio.conf please? Also FYI any change to our conf will require a service restart.
paths: {
local: “/var/lib/dremio”
dist: “maprfs://data/chenlini/pdfs”
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: false
}
services.coordinator.master.embedded-zookeeper.enabled: false
zookeeper: “host1:5181/dremio2,host2:5181/dremio2,host3:5181/dremio2”
this is the full complete dremio.conf, I did not add any thing else.
now the error info is “no cluster identity found”.
Exception in thread “main” java.lang.NullPointerException: No Cluster Ident
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java
at com.dremio.dac.daemon.DremioDaemon.checkVersion(DremioDaemon.java:89
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:125)
This is likely due to failed start-up attempts. Can you wipe our metadata db? rm -rf /var/lib/dremio/*
when I execute “rm -rf /var/lib/dremio/*”
There is no “No Cluster Ident” error, but the log still shows:
DEBUG c.d.s.coordinator.zk.ZKClusterClient - Connect: host1:5181, zkRoot: dremio2,host2:5181/dremio2,host3:5181, clusterId: dremio
Hi, is there any idea about this problem? I just wan to generate a dremio cluster named ‘dremio2’. thanks.
Can you attach your full sever.log & server.out? Can you also check your external ZK logs to see if there are any errors?