Hello, help to solve two difficult problems encountered in practical use, thank you.
Version: dremio 3.1.0
- not support mongodb Sharded cluster
2019-01-30 10:52:43,731 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - Created connection to [address=10.3.3.10:2000, user=mongodb].
2019-01-30 10:52:43,731 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - Number of open connections 1.
2019-01-30 10:52:43,731 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - MongoClientOptions:
MongoClientOptions{description=‘null’, readPreference=primary, writeConcern=WriteConcern{w=1, wtimeout=0, fsync=false, j=false, codecRegistry=org.bson.codecs.configuration.ProvidersCodecRegistry@9bbd3d59, minConnectionsPerHost=0, maxConnectionsPerHost=100, threadsAllowedToBlockForConnectionMultiplier=5, serverSelectionTimeout=2000, maxWaitTime=120000, maxConnectionIdleTime=0, maxConnectionLifeTime=0, connectTimeout=10000, socketTimeout=0, socketKeepAlive=false, sslEnabled=false, sslInvalidHostNamesAllowed=false, alwaysUseMBeans=false, heartbeatFrequency=10000, minHeartbeatFrequency=500, heartbeatConnectTimeout=20000, heartbeatSocketTimeout=20000, localThreshold=15, requiredReplicaSetName=‘null’, dbDecoderFactory=com.mongodb.DefaultDBDecoder$1@79a47e2e, dbEncoderFactory=com.mongodb.DefaultDBEncoder$1@273f7da9, socketFactory=javax.net.DefaultSocketFactory@73430433, cursorFinalizerEnabled=true, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitQueueSize=500, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, keepAlive=false, receiveBufferSize=0, sendBufferSize=0}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=20000, readTimeoutMS=20000, keepAlive=false, receiveBufferSize=0, sendBufferSize=0}}
2019-01-30 10:52:43,732 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - MongoCredential:MongoCredential{mechanism=null, userName=‘mongodb’, source=‘im’, password=, mechanismProperties={}}
2019-01-30 10:52:43,772 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - Created connection to [address=mongodb-1:2002, user=mongodb].
2019-01-30 10:52:43,772 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - Number of open connections 2.
2019-01-30 10:52:43,772 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - MongoClientOptions:
MongoClientOptions{description=‘null’, readPreference=primary, writeConcern=WriteConcern{w=1, wtimeout=0, fsync=false, j=false, codecRegistry=org.bson.codecs.configuration.ProvidersCodecRegistry@9bbd3d59, minConnectionsPerHost=0, maxConnectionsPerHost=100, threadsAllowedToBlockForConnectionMultiplier=5, serverSelectionTimeout=2000, maxWaitTime=120000, maxConnectionIdleTime=0, maxConnectionLifeTime=0, connectTimeout=10000, socketTimeout=0, socketKeepAlive=false, sslEnabled=false, sslInvalidHostNamesAllowed=false, alwaysUseMBeans=false, heartbeatFrequency=10000, minHeartbeatFrequency=500, heartbeatConnectTimeout=20000, heartbeatSocketTimeout=20000, localThreshold=15, requiredReplicaSetName=‘null’, dbDecoderFactory=com.mongodb.DefaultDBDecoder$1@79a47e2e, dbEncoderFactory=com.mongodb.DefaultDBEncoder$1@273f7da9, socketFactory=javax.net.DefaultSocketFactory@73430433, cursorFinalizerEnabled=true, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitQueueSize=500, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, keepAlive=false, receiveBufferSize=0, sendBufferSize=0}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=20000, readTimeoutMS=20000, keepAlive=false, receiveBufferSize=0, sendBufferSize=0}}
2019-01-30 10:52:43,773 [Plugin Startup: mon] INFO c.d.p.m.c.MongoConnectionManager - MongoCredential:MongoCredential{mechanism=null, userName=‘mongodb’, source=‘im’, password=, mechanismProperties={}}
2019-01-30 10:52:45,773 [Plugin Startup: mon] ERROR c.d.p.mongo.MongoDocumentHelper - Failed to run commandDocument{{ping=1}}, exception : com.mongodb.MongoTimeoutException: Timed out after 2000 ms while waiting for a server that matches PrimaryServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=mongodb-1:2002, type=UNKNOWN, state=CONNECTING}]
2019-01-30 10:52:45,774 [Plugin Startup: mon] ERROR c.d.p.mongo.MongoDocumentHelper - Mongo command returned with invalid return code (not OK), Failed to ping mongo cluster : { “Exception” : “com.mongodb.MongoTimeoutException: Timed out after 2000 ms while waiting for a server that matches PrimaryServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=mongodb-1:2002, type=UNKNOWN, state=CONNECTING}]” }
2019-01-30 10:52:45,774 [Plugin Startup: mon] WARN c.d.p.mongo.metadata.MongoTopology - User does not have access to MongoD instance ‘mongodb-1:2002’ in a replica set, this will slow down performance. Entering single thread mode
2019-01-30 10:52:45,780 [qtp376890625-147] INFO c.d.exec.catalog.CatalogServiceImpl - User Error Occurred [ErrorId: a335845a-9e39-4d1c-b7db-e5514d1735b6]
com.dremio.common.exceptions.UserException: Failure while configuring source [mon]
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.exec.catalog.CatalogServiceImpl.createOrUpdateSource(CatalogServiceImpl.java:639) [dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.exec.catalog.CatalogServiceImpl.createSource(CatalogServiceImpl.java:378) [dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.exec.catalog.CatalogServiceImpl.access$600(CatalogServiceImpl.java:102) [dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.exec.catalog.CatalogServiceImpl$SourceModifier.createSource(CatalogServiceImpl.java:972) [dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.exec.catalog.CatalogImpl.createSource(CatalogImpl.java:519) [dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.exec.catalog.DelegatingCatalog.createSource(DelegatingCatalog.java:189) [dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.dac.service.source.SourceService.registerSourceWithRuntime(SourceService.java:157) [dremio-dac-backend-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.dac.service.source.SourceService.registerSourceWithRuntime(SourceService.java:148) [dremio-dac-backend-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.dac.service.source.SourceService.registerSourceWithRuntime(SourceService.java:144) [dremio-dac-backend-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.dac.resource.PutSourceResource.putSource(PutSourceResource.java:80) [dremio-dac-backend-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_162]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_162]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [jersey-server-2.25.1.jar:na]
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [jersey-container-servlet-core-2.25.1.jar:na]
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [jersey-container-servlet-core-2.25.1.jar:na]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) [jetty-servlets-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:301) [jetty-servlets-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [jetty-servlet-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) [jetty-server-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) [jetty-io-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [jetty-util-9.2.26.v20180806.jar:9.2.26.v20180806]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [jetty-util-9.2.26.v20180806.jar:9.2.26.v20180806]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Cannot parse version 3.4.10-2.10
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:500) ~[guava-20.0.jar:na]
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:401) ~[guava-20.0.jar:na]
at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:83) ~[guava-20.0.jar:na]
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:68) ~[guava-20.0.jar:na]
at com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:104) ~[guava-20.0.jar:na]
at com.dremio.exec.catalog.CatalogServiceImpl.createOrUpdateSource(CatalogServiceImpl.java:627) [dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
… 55 common frames omitted
Caused by: java.lang.IllegalArgumentException: Cannot parse version 3.4.10-2.10
at com.dremio.plugins.Version.parse(Version.java:60) ~[dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.plugins.mongo.metadata.MongoVersion.getVersionForConnection(MongoVersion.java:52) ~[dremio-extra-plugin-mongo-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.plugins.mongo.metadata.MongoTopology.getMinVersion(MongoTopology.java:220) ~[dremio-extra-plugin-mongo-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.plugins.mongo.metadata.MongoTopology.getClusterTopology(MongoTopology.java:205) ~[dremio-extra-plugin-mongo-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.plugins.mongo.connection.MongoConnectionManager.getTopology(MongoConnectionManager.java:174) ~[dremio-extra-plugin-mongo-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.plugins.mongo.connection.MongoConnectionManager.validateConnection(MongoConnectionManager.java:310) ~[dremio-extra-plugin-mongo-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.plugins.mongo.connection.MongoConnectionManager.connect(MongoConnectionManager.java:145) ~[dremio-extra-plugin-mongo-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.plugins.mongo.MongoStoragePlugin.start(MongoStoragePlugin.java:127) ~[dremio-extra-plugin-mongo-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.exec.catalog.ManagedStoragePlugin$1.run(ManagedStoragePlugin.java:246) ~[dremio-sabot-kernel-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.concurrent.RenamingRunnable.run(RenamingRunnable.java:36) ~[dremio-common-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.concurrent.SingletonRunnable.run(SingletonRunnable.java:41) ~[dremio-common-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.concurrent.SafeRunnable.run(SafeRunnable.java:40) ~[dremio-common-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
at com.dremio.concurrent.Runnables$1.run(Runnables.java:45) ~[dremio-common-3.1.0-201901172111160703-dc6f6e5.jar:3.1.0-201901172111160703-dc6f6e5]
2)MySQL tinyint
com.dremio.common.exceptions.UserException: Invalid value for boolean: ‘2’
MySQL tinyint --> integer, but --> boolean
Tinyint should be converted to integer instead of Boolean.