We cannot connect to MongoDB (version 4.0.6-3) which is running on a DigitalOcean Droplet from Dremio (version 3.1.9) anymore. It used to work, but https was added to the Droplet, which might have caused the connection issue now? When we use RoboMongo, it connects without problems to this MongoDB on the Digital Ocean Droplet. Please advise.
Thanks!
Regards
Willem
Hello @willemderu,
Can you share your configuration for the Mongo source?
https://docs.dremio.com/data-sources/mongo.html
What is the specific error that youâre seeing?
Hi @ben , thanks for the assistance, it is appreciated very much. Apologies, I should have added more details to the original topic question above, thanks for you patience 
We have previously used the instructions in the document that you have linked to as follows:
We are using the Dremio MongoDB Source and have set the Host to:
188.226.181.157 and Port 27017
For Authentication, we use âMaster Credentialsâ
Username: StratSmart and Password: stratpwd (I will change this password after getting it going. The db now only contains test data.)
For the Authentication Database we have the following: formioapp
Other than the above, we have just kept the default settings in the Dremio MongoDB Source.
If the âEncrypt connectionâ setting is checked, it gives the following error:
Failure creating/updating source [StratSmart MongoDB]
If the âEncrypt connectionâ setting is not checked, it then gives the following error:
Could not find/access authentication database formioapp
(The above works when linking to the db with RoboMongo (we specify the Auth Mechanism as SCRAM-SHA-1))
I can provide the log output as well, if required.
Thanks!
Regards
Willem
@willemderu, can you try the connection and follow the logs as you attempt this? I believe they should give a more verbose error message.
@ben, below are the log output for reference. Maybe I should try to move to a previous version of MongoDB. (it is strange though that robomongo and âNoSQLBooster for MongoDBâ both connect to MongoDB without any issues at all)
2019-04-24 22:18:13,856 [qtp1249000159-215] INFO c.d.e.catalog.ManagedStoragePlugin - User Error Occurred [ErrorId: 2967ab76-2e7a-48fb-be1e-d565912fcc5f]
com.dremio.common.exceptions.UserException: The source [âStratSmart MongoDBâ] is currently unavailable. Info: [[Message{level=ERROR, msg=For input string: â6-3â}]].
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-3.1.10-201904162146020182-adf690d.jar:3.1.10-201904162146020182-adf690d]
at com.dremio.exec.catalog.ManagedStoragePlugin.checkState(ManagedStoragePlugin.java:356) [dremio-sabot-kernel-3.1.10-201904162146020182-adf690d.jar:3.1.10-201904162146020182-adf690d]
at com.dremio.exec.catalog.ManagedStoragePlugin.unwrap(ManagedStoragePlugin.java:600) [dremio-sabot-kernel-3.1.10-201904162146020182-adf690d.jar:3.1.10-201904162146020182-adf690d]
at com.dremio.exec.catalog.CatalogServiceImpl.getSource(CatalogServiceImpl.java:867) [dremio-sabot-kernel-3.1.10-201904162146020182-adf690d.jar:3.1.10-201904162146020182-adf690d]
at com.dremio.dac.service.source.SourceService.listSource(SourceService.java:374) [dremio-dac-backend-3.1.10-201904162146020182-adf690d.jar:3.1.10-201904162146020182-adf690d]
at com.dremio.dac.resource.SourceResource.getSource(SourceResource.java:159) [dremio-dac-backend-3.1.10-201904162146020182-adf690d.jar:3.1.10-201904162146020182-adf690d]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_201]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_201]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_201]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_201]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [jersey-container-servlet-core-2.25.1.jar:na]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) [jetty-servlets-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:365) [jetty-servlets-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) [jetty-io-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [jetty-util-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [jetty-util-9.2.26.v20180806.jar:9.2.26.v20180806]
at java.lang.Thread.run(Unknown Source) [na:1.8.0_201]
Thanks @willemderu, unfortunately the error is message is frustratingly generic. A couple of questions:
Thanks for the assistance @ben It is much appreciated. I have done a lot of experimentation the past while, (without success, though). To answer your questions: (1) The mongo instance is not a replica set. (2) It seems that it might not be related to moving to https. After removing the SSL Certificate to test, Dremio still did not want to connect to MongoDB. (although client apps such as MongoChef (Robo 3T) and noSQLBooster for MongoDB, connects without problems from the same environment)
I see in this thread in the Dremio Community Forum: Percona Mongodb 4.0.9-4 null pointer that Mongodb 4 is not supported yet by Dremio? Our issue might be related to this? (We cannot move back to MongoDB 3 though) What is the approx roadmap for Mongodb 4 support and what is the approx time-frames for moving the MongoDB connector out of Beta?
Thanks!
Just for some more feedback, we spun up an experimental Digital Ocean Droplet and installed MongoDB 3.6.12. Dremio connected successfully to this MongoDB instance.
However, it is not viable to use MongoDB 3 because MongoDB 4 (as well the the Percona MongoDB 4 builds) contains functionality that is not available in MongoDB 3.
Thanks!
Willem
Just for feedback, I have managed to connect to MongoDB 4 with Dremio. I am not sure how it started working though. (I have not upgraded my version of Dremio yet when it started working again.) However, just providing feedback here that Dremio can connect to MongoDB 4 as well. Thanks!
Thanks for reporting your testing efforts @willemderu,
What version of Dremio did you connect to Mongo?
Hi @ben , I am currently using Dremio Build 3.2.3, which connects to MongoDB 4 without a problem.
(This is probably the wrong place to say this, but I just want to say that Dremio is awesome. I have tried many other alternatives to connect to and query MongoDB, but the features provided by Dremio is spot-on for our requirements. Thanks also for the Community Edition, which provides a very accessible and realistic way of getting up to speed with Dremio and getting real value from Dremio.)