Hello,
I’m using Elasticsearch 6.5.1 (dockerized) and trying to add it as datasource from Dremio (build 3.0.6-201812082352540436-1f684f9) , dockerized also
Elastic instance is not behind a loadbalancer
querying it from outside http://es-host:9200/_search?pretty it’s working fine
Logs :
2019-01-13 13:04:57,135 [Plugin Startup: ES] INFO c.d.p.elastic.ElasticConnectionPool - User Error Occurred [ErrorId: 34b5ed6a-d1e0-40a3-8f8f-cdeca1068ed5]
com.dremio.common.exceptions.UserException: Encountered a problem while executing com.dremio.plugins.elastic.ElasticActions$NodesInfo@487b923c. Cannot gather Elasticsearch nodes information. Please make sure that the user has [cluster:monitor/nodes/info] privilege.
any idea to solve it ?
Thanks in advance
1 Like
Hi @gitrekm
Are you able to run the curl command successfully from the Dremio coordinator running inside the docker?
Thanks
@balaji.ramaswamy
Hi @balaji.ramaswamy
from inside the container it’s failing "no route to host " you’re right
In fact intercontainer networking on the same docker host is disabled by default
to enable it, the following commands need to be executed :
firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 4 -i docker0 -j ACCEPT
firewall-cmd --reload
systemctl restart docker
Thanks for the support
Thanks for the update @gitrekm, glad it works now
Kindly reach out if you have any other questions
Thanks
@balaji.ramaswamy
@balaji.ramaswamy @gitrekm
I’ve run into the same error on MacOs with the following docker containers:
Elasticsearch 6.8.0 and Dremio (docker version 18.09.3 )
ES is accessible outside the container with: curl --location --request GET ‘localhost:9200/_all?pretty’
Also, adding dremio to the existing ES network did not work for me e.g. docker network esnet dremio.
Any thoughts?
@christopherrbyrd,
Looked at your logfile and it looks like there is a connection refused. From the Dremio coordinator can you try “telnet es-host port”?
2020-02-24 03:35:24,935 [Plugin Startup: ShowMe] INFO c.d.p.elastic.ElasticConnectionPool - User Error Occurred [ErrorId: 8ec4b6d5-3be3-4b16-8e73-cf79bfd69098]
com.dremio.common.exceptions.UserException: Encountered a problem while executing com.dremio.plugins.elastic.ElasticActions$NodesInfo@76485c1e. Cannot gather Elasticsearch nodes information. Please make sure that the user has [cluster:monitor/nodes/info] privilege.
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:776) ~[dremio-common-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticConnectionPool.addContextAndThrow(ElasticConnectionPool.java:481) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticConnectionPool.access$500(ElasticConnectionPool.java:91) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticConnectionPool$ElasticConnection.executeAndHandleResponseCode(ElasticConnectionPool.java:656) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticConnectionPool.getHostList(ElasticConnectionPool.java:379) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticConnectionPool.updateClients(ElasticConnectionPool.java:240) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticConnectionPool.connect(ElasticConnectionPool.java:230) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticsearchStoragePlugin.start(ElasticsearchStoragePlugin.java:522) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.exec.catalog.ManagedStoragePlugin$1.run(ManagedStoragePlugin.java:257) [dremio-sabot-kernel-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.concurrent.RenamingRunnable.run(RenamingRunnable.java:36) [dremio-common-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.concurrent.SingletonRunnable.run(SingletonRunnable.java:41) [dremio-common-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.concurrent.SafeRunnable.run(SafeRunnable.java:40) [dremio-common-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.concurrent.Runnables$1.run(Runnables.java:45) [dremio-common-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
Caused by: javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused (Connection refused)
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287) ~[jersey-client-2.25.1.jar:na]
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252) ~[jersey-client-2.25.1.jar:na]
at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:701) ~[jersey-client-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:228) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:697) ~[jersey-client-2.25.1.jar:na]
at com.dremio.plugins.elastic.ElasticActions$NodesInfo.getResult(ElasticActions.java:147) ~[dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
at com.dremio.plugins.elastic.ElasticConnectionPool$ElasticConnection.executeAndHandleResponseCode(ElasticConnectionPool.java:654) [dremio-elasticsearch-plugin-4.0.1-201909191652190301-211720e.jar:4.0.1-201909191652190301-211720e]
... 9 common frames omitted
@balaji.ramaswamy,
I’m able to connect and run queries/jobs against the cluster with no problem e.g. from Kibana, Curl, SDK, etc… However, I don’t have telnet or ssh ports open on the local instance. I use Docker exec… Should I open any specific port(s) in order for Dremio to successfully connect? If so, which?
Note: the dremio container I’m running only has ssh.
Update: I’m going to enable the “xpack.security” layer, since “no authentication” fails on dremio.
@balaji.ramaswamy,
Issue resolved by adding the dremio container to the proper docker network e.g.
docker network connect dremio