Unable to connect to NAS

1

Running 3.0.6 community locally on CentOS 7.3. Trying to connect to a simple Desktop folder of sample data via the NAS connector. On the UI, I get error Failure while configuring source, and the log reports Caused by: java.lang.Exception: Unavailable: file /home/user1/Desktop/sample_data does not exist. But, of course, it does. Same result if I point at the file itself or the directory containing it.

The permissions on the are -rw-rw-r--.

I see others have had this problem, but no solutions arrived at in forum. Thanks.

ETA: the sample file is tiny so it is definitely not a memory or other size item

2

Hi @can1

On your NAS source , did you give the path of your desktop? Then promote sample_data using the right extreme button, see screenschot below. Also what type of files do you have inside sample_data? Can we have the server.log?

image
image.png2244×448 30.9 KB

3

Hi @balaji.ramaswamy. I tried giving the path to my Desktop only. Same error. Log is below.

2019-01-14 15:37:40,590 [qtp1023763480-118] INFO  c.d.exec.catalog.CatalogServiceImpl - User Error Occurred [ErrorId: 1929d1bd-d41e-4d53-8ab1-0516c9ecb9f2]
com.dremio.common.exceptions.UserException: Failure while configuring source [desktop_nas]
	at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.exec.catalog.CatalogServiceImpl.createOrUpdateSource(CatalogServiceImpl.java:634) [dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.exec.catalog.CatalogServiceImpl.createSource(CatalogServiceImpl.java:378) [dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.exec.catalog.CatalogServiceImpl.access$600(CatalogServiceImpl.java:100) [dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.exec.catalog.CatalogServiceImpl$SourceModifier.createSource(CatalogServiceImpl.java:954) [dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.exec.catalog.CatalogImpl.createSource(CatalogImpl.java:507) [dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.exec.catalog.DelegatingCatalog.createSource(DelegatingCatalog.java:189) [dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.dac.service.source.SourceService.registerSourceWithRuntime(SourceService.java:157) [dremio-dac-backend-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.dac.service.source.SourceService.registerSourceWithRuntime(SourceService.java:148) [dremio-dac-backend-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.dac.service.source.SourceService.registerSourceWithRuntime(SourceService.java:144) [dremio-dac-backend-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.dac.resource.PutSourceResource.putSource(PutSourceResource.java:80) [dremio-dac-backend-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.25.1.jar:na]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.25.1.jar:na]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.25.1.jar:na]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.25.1.jar:na]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.25.1.jar:na]
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jersey-common-2.25.1.jar:na]
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [jersey-server-2.25.1.jar:na]
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [jersey-container-servlet-core-2.25.1.jar:na]
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [jersey-container-servlet-core-2.25.1.jar:na]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [jersey-container-servlet-core-2.25.1.jar:na]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [jersey-container-servlet-core-2.25.1.jar:na]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [jersey-container-servlet-core-2.25.1.jar:na]
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) [jetty-servlet-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) [jetty-servlet-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) [jetty-servlets-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:301) [jetty-servlets-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [jetty-servlet-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) [jetty-server-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) [jetty-io-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [jetty-util-9.2.22.v20170606.jar:9.2.22.v20170606]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [jetty-util-9.2.22.v20170606.jar:9.2.22.v20170606]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]
Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: Unavailable: File /home/user1/Desktop does not exist
	at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:500) ~[guava-20.0.jar:na]
	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:401) ~[guava-20.0.jar:na]
	at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:83) ~[guava-20.0.jar:na]
	at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:68) ~[guava-20.0.jar:na]
	at com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:104) ~[guava-20.0.jar:na]
	at com.dremio.exec.catalog.CatalogServiceImpl.createOrUpdateSource(CatalogServiceImpl.java:622) [dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	... 55 common frames omitted
Caused by: java.lang.Exception: Unavailable: File /home/user1/Desktop does not exist
	at com.dremio.exec.catalog.ManagedStoragePlugin$1.run(ManagedStoragePlugin.java:252) ~[dremio-sabot-kernel-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.concurrent.RenamingRunnable.run(RenamingRunnable.java:36) ~[dremio-common-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.concurrent.SingletonRunnable.run(SingletonRunnable.java:41) ~[dremio-common-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.concurrent.SafeRunnable.run(SafeRunnable.java:40) ~[dremio-common-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
	at com.dremio.concurrent.Runnables$1.run(Runnables.java:45) ~[dremio-common-3.0.6-201812082352540436-1f684f9.jar:3.0.6-201812082352540436-1f684f9]
4

Hi @balaji.ramaswamy. Tried to reply with the server.log but it got tossed into moderation, probably due to length. if you can fish it out you can see. But, I tried putting the path to the desktop only and got the same error.

5

Screenshot%20from%202019-01-14%2015-30-26
Screenshot from 2019-01-14 15-30-26.png849×286 15.2 KB

Here is photo on the UI error

6

Hi @can

I am a little confused,

First time you had the message “/home/user1/Desktop/sample_data” second time it is only “/home/user1/Desktop”

Can you please open terminal on your mac and run the below command?

ls -ltrh /home/user1/Desktop/

Thanks
@balaji.ramaswamy

7

Hi @balaji.ramaswamy. This first time I was trying to connect directly to my directory of sample data, /home/user1/Desktop/sample_data. Then in your reply it seemed like you were advising me to connect NAS to the Desktop, /home/user1/Desktop, and then use Dremio UI to point to exact data. Either way, I get the same error on UI and in the logs. (Only difference is the the path in the logs – it’s whatever path I put in.)

In answer to your request:

[user1@localhost ~]$ ls -ltrh /home/user1/Desktop/
total 40K
drwxrwxr-x. 2 user1 user1   60 Jan 14 15:02 sample_data
8

Hi @can1

Would you be able to send me the output of

ps -ef | grep dremio

Thanks
@balaji.ramaswamy

9

Hi @balaji.ramaswamy, it’s

user1     2420  1982  0 16:17 pts/0    00:00:00 grep --color=auto dremio
dremio   15070     1  1 14:57 ?        00:01:01 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/opt/dremio/lib -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/dremio/server.gc -Ddremio.log.path=/var/log/dremio -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/dremio -Dio.netty.maxDirectMemory=0 -DMAPR_IMPALA_RA_THROTTLE -DMAPR_MAX_RA_STREAMS=400 -cp /etc/dremio:/opt/dremio/jars/*:/opt/dremio/jars/ext/*:/opt/dremio/jars/3rdparty/* com.dremio.dac.daemon.DremioDaemon
10

What happens if you do the below?

su - dremio
ls -ltrh /home/user1/Desktop/

11

@balaji.ramaswamy I get an authentication error with just the first line. I believe this makes me switch users? But I don’t have a user called dremio.

ETA at any rate it’s not clear what password to use, and they all fail – my root password, my user1 password, my dremio admin password.

12

Is your a RPM install which creates a dremio user implicitly, see below, dremio is running as dremio unix user

dremio 15070 1 1 14:57 ? 00:01:01 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/opt/dremio/lib -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/dremio/server.gc -Ddremio.log.path=/var/log/dremio -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/dremio -Dio.netty.maxDirectMemory=0 -DMAPR_IMPALA_RA_THROTTLE -DMAPR_MAX_RA_STREAMS=400 -cp /etc/dremio:/opt/dremio/jars/:/opt/dremio/jars/ext/:/opt/dremio/jars/3rdparty/* com.dremio.dac.daemon.DremioDaemon

Can you please send output of below?

ls -ld /home/user1/Desktop/

13

@balaji.ramaswamy it is an RPM install so that makes sense. I don’t know what the password would be though.

The output is

[user1@localhost /]$ ls -ld /home/user1/Desktop
drwxr-xr-x. 3 user1 user1 125 Jan 14 16:17 /home/user1/Desktop
14

Try this,

sudo su -

su - dremio -c "ls -ltrh /home/user1/Desktop"
15

@balaji.ramaswamy unfortunately that didn’t work either. i get

[user1@localhost /]$ sudo su -
[sudo] password for user1: 
Last login: Mon Jan 14 16:29:51 EST 2019 on pts/0
[root@localhost ~]# su - dremio -c "ls -ltrh /home/user1/Desktop"
This account is currently not available.

So I can log in successfully as user1 but can’t access dremio. Since the dremio account is created automatically, is there any documentation on what its password is or how to access it?

16

Hi @can1

This is the dremio unix account. if you have the root login, you can simply so

su - dremio
and then try to access the folder,

you may have to vi /etc/passwd and remove nologin and set a home directory and shell type like bash

Note; You can quickly try another thing, stop dremio, download the tar ball and install as user1 and try to add the source

Thanks
@balaji.ramaswamy

17

Hello @balaji.ramaswamy, even with the root login I get:

[root@localhost ~]$ su - dremio
Last login: Mon Jan 14 17:12:52 EST 2019 on pts/1
This account is currently not available.

And even when I am logged in as root, I get the exact same set of errors when trying to choose my Desktop directory as a NAS source.

18

Hi @can1

Try this

  1. Shutdown all Dremio services on the host
  2. Download Dremio tar ball from dremio.com and install as user “user1”
  3. Start dremio as “user1”
  4. Add source and see if it suceeds

Thanks
@balaji.ramaswamy

19

Hi @balaji.ramaswamy, I made a new post here on this topic with cleaner examples and more info.

But the upshot is, I installed from a tarball as suggested and got the exact same problem.