Error when deleting YARN provisioning due to missing Kafka class on Windows

1

When trying to remove a (wrong) configured YARN cluster: status remained requested - starting for a very long time.

I cannot delete it (by clicking the trash-icon) and get the following error:

2019-01-09 17:11:54,874 [qtp1456572880-168] ERROR c.d.d.server.GenericExceptionMapper - Unexpected exception when processing DELETE http://localhost:9047/apiv2/provision/cluster/2885fb83-3e47-4758-86c0-926acf3e879a : java.lang.NoClassDefFoundError: kafka/api/OffsetRequest
java.lang.NoClassDefFoundError: kafka/api/OffsetRequest
at org.apache.twill.internal.kafka.client.SimpleKafkaConsumer$SimplePreparer.addFromBeginning(SimpleKafkaConsumer.java:205) ~[dremio-twill-shaded-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
at org.apache.twill.internal.AbstractTwillController.doStartUp(AbstractTwillController.java:106) ~[dremio-twill-shaded-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
at org.apache.twill.yarn.YarnTwillController.doStartUp(YarnTwillController.java:119) ~[dremio-twill-shaded-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
at org.apache.twill.internal.AbstractZKServiceController.startUp(AbstractZKServiceController.java:75) ~[dremio-twill-shaded-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
at org.apache.twill.internal.AbstractExecutionServiceController$ServiceDelegate.startUp(AbstractExecutionServiceController.java:187) ~[dremio-twill-shaded-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
at twill.com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ~[dremio-twill-shaded-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: java.lang.ClassNotFoundException: kafka.api.OffsetRequest
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_181]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_181]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[na:1.8.0_181]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_181]
... 7 common frames omitted
2

Have you tried restarting the Dremio service? Did a daemon job actually start in YARN?

3

Try this @orlandob

  1. Refresh URL
  2. Click delete icon one more time
  3. Refresh URL

Thanks
@balaji.ramaswamy

4

Hi @balaji.ramaswamy,

it worked for the deletion after refreshing multiple times.

5

Hi @anthony,

In fact nothing happened on YARN. Even a couple of hours later: the status is still starting and requested.