Dremio is exiting. Failure while starting services

1

What is a good resource to find out why Dremio is failing to start? Instance has been running for over 3 month. Crashed today.

Dremio is exiting. Failure while starting services.
java.lang.NoSuchMethodError: io.grpc.protobuf.lite.ProtoLiteUtils.metadataMarshaller(Lcom/google/protobuf/MessageLite;)Lio/grpc/Metadata$BinaryMarshaller;
at com.dremio.service.jobs.JobsRpcUtils.(JobsRpcUtils.java:61)
at com.dremio.service.jobs.JobsServer.start(JobsServer.java:91)
at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)
at com.dremio.service.ServiceRegistry.start(ServiceRegistry.java:88)
at com.dremio.service.SingletonRegistry.start(SingletonRegistry.java:33)
at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:195)
at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:201)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:146)

2

Did you upgrade to a newer version perhaps?

3

Nothing was done. Running for 3 months, Dremio crashed. I did notice

2020-07-16T20:30:33.523+0000: 6113108.203: [Full GC (Allocation Failure) 7637M->7637M(8192M), 6.5979688 secs]
[Eden: 0.0B(384.0M)->0.0B(384.0M) Survivors: 0.0B->0.0B Heap: 7637.4M(8192.0M)->7637.4M(8192.0M)], [Metaspace: 195051K->195051K(1331200K)]
[Times: user=9.61 sys=0.00, real=6.60 secs]
2020-07-16T20:30:40.121+0000: 6113114.801: [Full GC (Allocation Failure)

I am going to try restoring the database from a previous backup

4

Restoring DB database did not work. I didn’t think so. Looks like java cannot find a module. Next step is to reinstall java

5

Issue has been identified and resolved.

Tableau bug caused the Coordinator server to run out of memory, corrupting the Dremio software. Reinstalled the Dremio software resolved the issue. Strangely, the Dremio software on the Executor servers was also corrupted. Dremio software had to be reinstalled on those servers as well

https://community.tableau.com/s/question/0D54T00000C6ftvSAB/select-sum-1-as-col-from-uattablename

6

For future searches…below is the error in the server log. Notice Dremio application could not find a method.

Dremio is exiting. Failure while starting services.
java.lang.NoSuchMethodError: io.grpc.protobuf.lite.ProtoLiteUtils.metadataMarshaller(Lcom/google/protobuf/MessageLite;)Lio/grpc/Metadata$BinaryMarshaller;

    at com.dremio.service.jobs.JobsRpcUtils.<clinit>(JobsRpcUtils.java:61)

    at com.dremio.service.jobs.JobsServer.start(JobsServer.java:91)

    at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)
    at com.dremio.service.ServiceRegistry.start(ServiceRegistry.java:88)
    at com.dremio.service.SingletonRegistry.start(SingletonRegistry.java:33)
    at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:195)
    at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:201)
    at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:146)