Sharing Cluster Metadata on S3

1

Guys, I gonna add one more node on my single node deployment, but I have some questions.

In previous versions, I’ve done this type of deployment (1 coordinator + 2 executors) but I don’t remember to have done any type of network shared folder.

There is another way to share metadata among nodes, like using S3?

2

Hello @allan.sene, the shared network folder is used for the HA configuration with 2 master coordinators. If you are just deploying 1 coordinator + n executors, you don’t need it. You need a reliable volume connected to the coordinator host, where Dremio can persist its metadata (the db directory and it’s contents).

3

Thanks, @ben! Anyway… I’m getting this error when trying to start this new executor node:

Dremio is exiting. Failure while starting services.
java.lang.RuntimeException: Failure while attempting to create com.dremio.service.users.SimpleUserService.
	at com.dremio.service.BinderImpl$InjectableReference.get(BinderImpl.java:427)
	at com.dremio.service.BinderImpl.lookup(BinderImpl.java:109)
	at com.dremio.service.BinderImpl$DeferredProvider.get(BinderImpl.java:83)
	at com.dremio.exec.server.ContextService.newSabotContext(ContextService.java:225)
	at com.dremio.exec.server.ContextService.start(ContextService.java:168)
	at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)
	at com.dremio.dac.daemon.NonMasterSingletonRegistry.start(NonMasterSingletonRegistry.java:54)
	at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:183)
	at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:189)
	at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:102)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.dremio.service.BinderImpl$InjectableReference.get(BinderImpl.java:421)
	... 9 more
Caused by: java.lang.NullPointerException: Unknown store creator com.dremio.service.users.SimpleUserService$UserGroupStoreBuilder
	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:864)
	at com.dremio.datastore.RemoteKVStoreProvider.getStore(RemoteKVStoreProvider.java:76)
	at com.dremio.service.users.SimpleUserService.<init>(SimpleUserService.java:96)
	... 14 more
4

@allan.sene, just making sure, but you have configured the executor appropriately:

services: {
  coordinator.enabled: false,
  coordinator.master.enabled: false,
  executor.enabled: true
}

Also, what user is the dremio service running as on that host and do they have the appropriate permissions to write to local directory configured in dremio.conf?

5

@ben I’m running as root cause, for security reasons, Dremio UI needs to expose on port 80.

The conf files are as the documentation recommends.

6

@allan.sene, unfortunately, Dremio’s startup errors are not very informative here. Are the required ports open to bi-directional traffic? In particular, 45678 will be needed for internode communications