Dremio is exiting . Failure while starting services

V5490 · August 29, 2019, 7:05am

Got a source code from github and tried to build through mvn. Build got success and after performing mvn exec:exec -pl dac/daemon, i got the issue as Dremio is exiting . Failure while starting services.

Below are the logs:

18:25:07.476 [main] ERROR ROOT - Dremio is exiting. Failure while starting services.

java.lang.RuntimeException: Failure while attempting to create com.dremio.dac.service.collaboration.CollaborationHelper.

at com.dremio.service.BinderImpl$InjectableReference.get(BinderImpl.java:427) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.service.BinderImpl.lookup(BinderImpl.java:109) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.service.BinderImpl$DeferredProvider.get(BinderImpl.java:83) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.dac.daemon.SampleDataPopulatorService.start(SampleDataPopulatorService.java:97) ~[dremio-dac-backend-3.3.2-201908142136370993-d60145d.jar:na]

at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.service.ServiceRegistry.start(ServiceRegistry.java:74) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.service.SingletonRegistry.start(SingletonRegistry.java:33) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:184) ~[dremio-dac-backend-3.3.2-201908142136370993-d60145d.jar:na]

at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:190) ~[dremio-dac-backend-3.3.2-201908142136370993-d60145d.jar:na]

at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:104) ~[classes/:na]

Caused by: java.lang.NullPointerException: Unable to find injectable based on javax.ws.rs.core.SecurityContext

at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:864) ~[guava-20.0.jar:na]

at com.dremio.service.BinderImpl.lookup(BinderImpl.java:107) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.service.BinderImpl$FinalResolver.getImplementation(BinderImpl.java:388) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.service.BinderImpl$InjectableReference$3.apply(BinderImpl.java:424) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.service.BinderImpl$InjectableReference$3.apply(BinderImpl.java:421) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.google.common.collect.Iterators$7.transform(Iterators.java:750) ~[guava-20.0.jar:na]

at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47) ~[guava-20.0.jar:na]

at com.google.common.collect.Iterators.addAll(Iterators.java:367) ~[guava-20.0.jar:na]

at com.google.common.collect.Lists.newArrayList(Lists.java:163) ~[guava-20.0.jar:na]

at com.google.common.collect.Iterables.castOrCopyToCollection(Iterables.java:344) ~[guava-20.0.jar:na]

at com.google.common.collect.Iterables.toArray(Iterables.java:323) ~[guava-20.0.jar:na]

at com.google.common.collect.Iterables.toArray(Iterables.java:319) ~[guava-20.0.jar:na]

at com.google.common.collect.FluentIterable.toArray(FluentIterable.java:779) ~[guava-20.0.jar:na]

at com.dremio.service.BinderImpl$InjectableReference.get(BinderImpl.java:425) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

… 9 common frames omitted

Dremio is exiting. Failure while starting services.

java.lang.RuntimeException: Failure while attempting to create com.dremio.dac.service.collaboration.CollaborationHelper.

at com.dremio.service.BinderImpl$InjectableReference.get(BinderImpl.java:427)

at com.dremio.service.BinderImpl.lookup(BinderImpl.java:109)

at com.dremio.service.BinderImpl$DeferredProvider.get(BinderImpl.java:83)

at com.dremio.dac.daemon.SampleDataPopulatorService.start(SampleDataPopulatorService.java:97)

at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)

at com.dremio.service.ServiceRegistry.start(ServiceRegistry.java:74)

at com.dremio.service.SingletonRegistry.start(SingletonRegistry.java:33)

at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:184)

at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:190)

at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:104)

Caused by: java.lang.NullPointerException: Unable to find injectable based on javax.ws.rs.core.SecurityContext

at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:864)

at com.dremio.service.BinderImpl.lookup(BinderImpl.java:107)

at com.dremio.service.BinderImpl$FinalResolver.getImplementation(BinderImpl.java:388)

at com.dremio.service.BinderImpl$InjectableReference$3.apply(BinderImpl.java:424)

at com.dremio.service.BinderImpl$InjectableReference$3.apply(BinderImpl.java:421)

at com.google.common.collect.Iterators$7.transform(Iterators.java:750)

at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)

at com.google.common.collect.Iterators.addAll(Iterators.java:367)

at com.google.common.collect.Lists.newArrayList(Lists.java:163)

at com.google.common.collect.Iterables.castOrCopyToCollection(Iterables.java:344)

at com.google.common.collect.Iterables.toArray(Iterables.java:323)

at com.google.common.collect.Iterables.toArray(Iterables.java:319)

at com.google.common.collect.FluentIterable.toArray(FluentIterable.java:779)

at com.dremio.service.BinderImpl$InjectableReference.get(BinderImpl.java:425)

… 9 more

[ ERROR ] Command execution failed.

org.apache.commons.exec.ExecuteException : Process exited with an error: 4 (Exit value: 4)

at org.apache.commons.exec.DefaultExecutor.executeInternal ( DefaultExecutor.java:404 )

at org.apache.commons.exec.DefaultExecutor.execute ( DefaultExecutor.java:166 )

at org.codehaus.mojo.exec.ExecMojo.executeCommandLine ( ExecMojo.java:804 )

at org.codehaus.mojo.exec.ExecMojo.executeCommandLine ( ExecMojo.java:751 )

at org.codehaus.mojo.exec.ExecMojo.execute ( ExecMojo.java:313 )

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo ( DefaultBuildPluginManager.java:137 )

at org.apache.maven.lifecycle.internal.MojoExecutor.execute ( MojoExecutor.java:210 )

at org.apache.maven.lifecycle.internal.MojoExecutor.execute ( MojoExecutor.java:156 )

at org.apache.maven.lifecycle.internal.MojoExecutor.execute ( MojoExecutor.java:148 )

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject ( LifecycleModuleBuilder.java:117 )

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject ( LifecycleModuleBuilder.java:81 )

at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build ( SingleThreadedBuilder.java:56 )

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute ( LifecycleStarter.java:128 )

at org.apache.maven.DefaultMaven.doExecute ( DefaultMaven.java:305 )

at org.apache.maven.DefaultMaven.doExecute ( DefaultMaven.java:192 )

at org.apache.maven.DefaultMaven.execute ( DefaultMaven.java:105 )

at org.apache.maven.cli.MavenCli.execute ( MavenCli.java:956 )

at org.apache.maven.cli.MavenCli.doMain ( MavenCli.java:288 )

at org.apache.maven.cli.MavenCli.main ( MavenCli.java:192 )

at sun.reflect.NativeMethodAccessorImpl.invoke0 ( Native Method )

at sun.reflect.NativeMethodAccessorImpl.invoke ( NativeMethodAccessorImpl.java:62 )

at sun.reflect.DelegatingMethodAccessorImpl.invoke ( DelegatingMethodAccessorImpl.java:43 )

at java.lang.reflect.Method.invoke ( Method.java:498 )

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced ( Launcher.java:282 )

at org.codehaus.plexus.classworlds.launcher.Launcher.launch ( Launcher.java:225 )

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode ( Launcher.java:406 )

at org.codehaus.plexus.classworlds.launcher.Launcher.main ( Launcher.java:347 )

[ INFO ] ------------------------------------------------------------------------

[ INFO ] BUILD FAILURE

[ INFO ] ------------------------------------------------------------------------

[ INFO ] Total time: 01:05 min

[ INFO ] Finished at: 2019-08-26T18:25:13+05:30

[ INFO ] ------------------------------------------------------------------------

[ ERROR ] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (default-cli) on project dremio-dac-daemon: Command execution failed. : Process exited with an error: 4 (Exit value: 4) -> [Help 1]

[ ERROR ]

[ ERROR ] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ ERROR ] Re-run Maven using the -X switch to enable full debug logging.

[ ERROR ]

[ ERROR ] For more information about the errors and possible solutions, please read the following articles:

[ ERROR ] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

QAs-MacBook-Air:dremio qa$

doron · August 29, 2019, 4:34pm

What version of the JDK are you using to compile?

V5490 · September 3, 2019, 4:06am

@doron
Thank you for looking up the issue and below are the java and maven details
Java:
openjdk version “1.8.0_212”
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b04)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b04, mixed mode)

Maven:
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-05T00:30:29+05:30)
Maven home: /Users/qa/Documents/apache-maven-3.6.1
Java version: 1.8.0_212, vendor: AdoptOpenJDK, runtime: /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
Default locale: en_IN, platform encoding: UTF-8
OS name: “mac os x”, version: “10.13.6”, arch: “x86_64”, family: “mac”

V5490 · September 4, 2019, 12:16pm

@doron
Thank you for looking up the issue and below are the java and maven details
Java:
openjdk version “1.8.0_212”
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b04)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b04, mixed mode)

Maven:
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-05T00:30:29+05:30)
Maven home: /Users/qa/Documents/apache-maven-3.6.1
Java version: 1.8.0_212, vendor: AdoptOpenJDK, runtime: /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
Default locale: en_IN, platform encoding: UTF-8
OS name: “mac os x”, version: “10.13.6”, arch: “x86_64”, family: “mac”

doron · September 4, 2019, 1:56pm

It seems likely its something to do with your OpenJDK as it works fine with the Oracle version. When I have some time I will try the same vendor (I assume this is the Homebrew provided Java?).

V5490 · September 5, 2019, 9:18am

@doron
once again thanks for helping me. Now i compiled with oracle version, build was success. But after performing mvn exec:exec -pl dac/daemon, i am getting “com.dremio.common.exceptions.UserException: No Cluster Identity found”

Java version “1.8.0_221”
Java™ SE Runtime Environment (build 1.8.0_221-b11)
Java HotSpot™ 64-Bit Server VM (build 25.221-b11, mixed mode)

Maven:
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-05T00:30:29+05:30)
Maven home: /Users/qa/Documents/apache-maven-3.6.1
Java version: 1.8.0_212, vendor: AdoptOpenJDK, runtime: /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
Default locale: en_IN, platform encoding: UTF-8
OS name: “mac os x”, version: “10.13.6”, arch: “x86_64”, family: “mac”

Below are the logs:

13:35:48.570 [main] INFO c.d.datastore.LocalKVStoreProvider - Starting LocalKVStoreProvider

13:35:49.711 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

13:35:50.654 [main] INFO c.g.c.h.f.g.GoogleHadoopFileSystemBase - GHFS version: 1.6.1-hadoop2

13:35:54.830 [main] INFO c.d.datastore.LocalKVStoreProvider - LocalKVStoreProvider is up

13:35:54.937 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopping LocalKVStoreProvider

13:35:54.988 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopped LocalKVStoreProvider

13:35:55.004 [main] ERROR ROOT - Dremio is exiting. Failure while starting services.

com.dremio.common.exceptions.UserException: No Cluster Identity found

at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:776) ~[dremio-common-3.3.2-201908142136370993-d60145d.jar:3.3.2-201908142136370993-d60145d]

at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:170) ~[classes/:na]

at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:160) ~[classes/:na]

at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:99) ~[classes/:na]

Dremio is exiting. Failure while starting services.

com.dremio.common.exceptions.UserException: No Cluster Identity found

at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:776)

at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:170)

at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:160)

at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:99)