Im currently unable to connect to AWS ELASTICSEARCH version 6.3.1. Using Dremio 3.0.1
Hi @50scratch
Currently that feature is in development and should have it out very soon. Within the next 2 minor releases. Will keep you posted
Thanks
@balaji.ramaswamy
Hi @50scratch
We have enhanced Dremio to support elasticsearch on AWS. If this is still a requirement, kindly download our latest version 3.0.5 from https://www.dremio.com/download/, upgrade, test and let us know
Thanks
@balaji.ramaswamy
upgraded to 3.0.5, but still seeing the same error. Please make sure that the user has [cluster:monitor/nodes/info] privilege
2018-12-10 12:16:10,227 [Plugin Startup: ONEMAPLIVE] INFO c.d.p.elastic.ElasticConnectionPool - User Error Occurred [ErrorId: 77cb3461-b218-42d1-b1c5-29e62b761d43]
com.dremio.common.exceptions.UserException: Encountered a problem while executing com.dremio.plugins.elastic.ElasticActions$NodesInfo@4ed35145. Cannot gather Elasticsearch nodes information. Please make sure that the user has [cluster:monitor/nodes/info] privilege.
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticConnectionPool.addContextAndThrow(ElasticConnectionPool.java:477) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticConnectionPool.access$500(ElasticConnectionPool.java:91) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticConnectionPool$ElasticConnection.executeAndHandleResponseCode(ElasticConnectionPool.java:652) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticConnectionPool.getHostList(ElasticConnectionPool.java:375) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticConnectionPool.updateClients(ElasticConnectionPool.java:236) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticConnectionPool.connect(ElasticConnectionPool.java:226) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticsearchStoragePlugin.start(ElasticsearchStoragePlugin.java:455) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.exec.catalog.ManagedStoragePlugin$1.run(ManagedStoragePlugin.java:246) [dremio-sabot-kernel-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.concurrent.RenamingRunnable.run(RenamingRunnable.java:36) [dremio-common-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.concurrent.SingletonRunnable.run(SingletonRunnable.java:41) [dremio-common-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.concurrent.SafeRunnable.run(SafeRunnable.java:40) [dremio-common-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.concurrent.Runnables$1.run(Runnables.java:45) [dremio-common-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
Caused by: javax.ws.rs.ProcessingException: java.net.UnknownHostException: https
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287) ~[jersey-client-2.25.1.jar:na]
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252) ~[jersey-client-2.25.1.jar:na]
at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:701) ~[jersey-client-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:228) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444) ~[jersey-common-2.25.1.jar:na]
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:697) ~[jersey-client-2.25.1.jar:na]
at com.dremio.plugins.elastic.ElasticActions$NodesInfo.getResult(ElasticActions.java:147) ~[dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
at com.dremio.plugins.elastic.ElasticConnectionPool$ElasticConnection.executeAndHandleResponseCode(ElasticConnectionPool.java:650) [dremio-elasticsearch-plugin-3.0.5-201812021947440442-9a23f45.jar:3.0.5-201812021947440442-9a23f45]
… 9 common frames omitted
Caused by: java.net.UnknownHostException: https
at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_191]
at java.net.PlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_191]
at java.net.SocksSocketImpl.connect(Unknown Source) ~[na:1.8.0_191]
at java.net.Socket.connect(Unknown Source) ~[na:1.8.0_191]
at sun.net.NetworkClient.doConnect(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.http.HttpClient.(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.http.HttpClient.New(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.http.HttpClient.New(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source) ~[na:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) ~[na:1.8.0_191]
at java.net.HttpURLConnection.getResponseCode(Unknown Source) ~[na:1.8.0_191]
at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:399) ~[jersey-client-2.25.1.jar:na]
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285) ~[jersey-client-2.25.1.jar:na]
… 18 common frames omitted
Hi @50scratch
Are you using the new source for Elasticsearch on AWS ? See screenshot below, you have to use the first one “Amazon Elasticsearch Service”
Thanks
@balaji.ramaswamy
Ye my mistake. I took a look at the new connector, but there is no option for host and user name and password. https://USER:Password@be09ee1655384371b576b3a3753c3749.us-west-2.aws.found.io
hey @50scratch are you using Amazon Elasticsearch Service or Elasticsearch’s own service?
elasticsearch-service hosted on amazon