I am getting the following error message when I try and connect to my AWS Elasticsearch database (v 5.5) with Dremio (v. 1.4.4) when clicking on schema (purple table icon):
‘Failure to retrieve metadata for table’
I am able to do a post request to _cluster/state/metadata and see the information, so I know the connection information is correct.
Can you please send us the profile for the failed job? We would need to take at the exact error to recommend next steps
Thanks,
@balaji.ramaswamy
Hi Balaji,
Sure. Attached is the profile, please let me know what other information you need.
Best,
Cory
45e964cf-4bbb-4f4c-a022-ae1d5f7c7a86.zip (2.53 KB)
Can you try to FORGET and REFRESH metadata and try to access the dataset again?
https://docs.dremio.com/sql-reference/sql-commands/datasets.html
Hi Balaji,
I did try those two commands already.
Thanks,
Cory
Can you please send output of below command
curl -XGET 'http://localhost:9200/_nodes'
Thanks,
@balaji.ramaswamy
Hi Balaji,
Our elasticsearch is on AWS, its not running on port 9200 on localhost, but when I query ‘_nodes’ for our elasticsearch server I get the following output:
{"_nodes":{"total":1,"successful":1,"failed":0},"cluster_name":"222122973777:eri-dm-logs","nodes":{"NFYtdVbCQr289gOPjHXNEw":{"name":"NFYtdVb","version":"5.5.2","build_hash":"Unknown","total_indexing_buffer":106502553,"roles":["master","data","ingest"],"os":{"refresh_interval_in_millis":1000,"available_processors":1,"allocated_processors":1},"process":{"refresh_interval_in_millis":1000,"id":5421,"mlockall":false},"jvm":{"pid":5421,"start_time_in_millis":1518189003296,"mem":{"heap_init_in_bytes":1073741824,"heap_max_in_bytes":1065025536,"non_heap_init_in_bytes":2555904,"non_heap_max_in_bytes":0,"direct_max_in_bytes":1065025536},"using_compressed_ordinary_object_pointers":"true"},"thread_pool":{"force_merge":{"type":"fixed","min":1,"max":1,"queue_size":-1},"fetch_shard_started":{"type":"scaling","min":1,"max":2,"keep_alive":"5m","queue_size":-1},"listener":{"type":"fixed","min":1,"max":1,"queue_size":-1},"index":{"type":"fixed","min":1,"max":1,"queue_size":200},"refresh":{"type":"scaling","min":1,"max":1,"keep_alive":"5m","queue_size":-1},"generic":{"type":"scaling","min":4,"max":128,"keep_alive":"30s","queue_size":-1},"warmer":{"type":"scaling","min":1,"max":1,"keep_alive":"5m","queue_size":-1},"search":{"type":"fixed","min":2,"max":2,"queue_size":1000},"flush":{"type":"scaling","min":1,"max":1,"keep_alive":"5m","queue_size":-1},"fetch_shard_store":{"type":"scaling","min":1,"max":2,"keep_alive":"5m","queue_size":-1},"management":{"type":"scaling","min":1,"max":5,"keep_alive":"5m","queue_size":-1},"get":{"type":"fixed","min":1,"max":1,"queue_size":1000},"bulk":{"type":"fixed","min":1,"max":1,"queue_size":200},"snapshot":{"type":"scaling","min":1,"max":1,"keep_alive":"5m","queue_size":-1}},"modules":[{"name":"aggs-matrix-stats","version":"5.5.2","description":"Adds aggregations whose input are a list of numeric fields and output includes a matrix.","classname":"org.elasticsearch.search.aggregations.matrix.MatrixAggregationPlugin","has_native_controller":false},{"name":"ingest-common","version":"5.5.2","description":"Module for ingest processors that do not require additional security permissions or have large dependencies and resources","classname":"org.elasticsearch.ingest.common.IngestCommonPlugin","has_native_controller":false},{"name":"lang-expression","version":"5.5.2","description":"Lucene expressions integration for Elasticsearch","classname":"org.elasticsearch.script.expression.ExpressionPlugin","has_native_controller":false},{"name":"lang-mustache","version":"5.5.2","description":"Mustache scripting integration for Elasticsearch","classname":"org.elasticsearch.script.mustache.MustachePlugin","has_native_controller":false},{"name":"lang-painless","version":"5.5.2","description":"An easy, safe and fast scripting language for Elasticsearch","classname":"org.elasticsearch.painless.PainlessPlugin","has_native_controller":false},{"name":"parent-join","version":"5.5.2","description":"This module adds the support parent-child queries and aggregations","classname":"org.elasticsearch.join.ParentJoinPlugin","has_native_controller":false},{"name":"percolator","version":"5.5.2","description":"Percolator module adds capability to index queries and query these queries by specifying documents","classname":"org.elasticsearch.percolator.PercolatorPlugin","has_native_controller":false},{"name":"reindex","version":"5.5.2","description":"The Reindex module adds APIs to reindex from one index to another or update documents in place.","classname":"org.elasticsearch.index.reindex.ReindexPlugin","has_native_controller":false},{"name":"transport-netty4","version":"5.5.2","description":"Netty 4 based transport implementation","classname":"org.elasticsearch.transport.Netty4Plugin","has_native_controller":false}],"ingest":{"processors":[{"type":"append"},{"type":"attachment"},{"type":"convert"},{"type":"date"},{"type":"date_index_name"},{"type":"dot_expander"},{"type":"fail"},{"type":"foreach"},{"type":"grok"},{"type":"gsub"},{"type":"join"},{"type":"json"},{"type":"kv"},{"type":"lowercase"},{"type":"remove"},{"type":"rename"},{"type":"script"},{"type":"set"},{"type":"sort"},{"type":"split"},{"type":"trim"},{"type":"uppercase"},{"type":"user_agent"}]}}}}
Let me know if this is what you needed.
Best,
Cory
We have released Dremio 3.0.5 that should address your AWS elasticsearch issue. Can you please upgrade, test and confirm
Thanks
@balaji.ramaswamy