Connect to OpenSearch (ex OpenDistro Elastic)

1

Hello everyone, I am trying to make a connection from Dremio to OpenSearch (ElasticSearch), but when using the ElasticSearch parameterization this gives an error:

"Failure getting Elastic health. null."

When reviewing the logs, you see this:

"Caused by: java.lang.RuntimeException: Dremio only supports Elasticsearch versions above 2.0.0. Found a node with version 1.2.1"

The information from opensearch is as follows:

{
  "name" : "node01",
  "cluster_name" : "test-opensearch",
  "cluster_uuid" : "BPVoFHuHbdy6ROZ4MxqWIA",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.2.1",
    "build_type" : "tar",
    "build_hash" : "e3a44fa71b290fb265a94ef4297f044b9a63a762",
    "build_date" : "2021-12-11T04:22:52.398139Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

This was tested with the dremio versions:
18.2
19.1

I appreciate your help with this integration.

Thanks

1 Like
2

@gcabrerap What version of OpenSearch is this/ Do all nodes run same version of Opensearch?

3

@balaji.ramaswamy The versión of opensearch is the latest + fix the Log4j , the cluster have 3 nodes with the latest versión.

4

@gcabrerap Currently Dremio only works with Elasticsearch, it seems like OpenSearch has some classes that Dremio is not handling, you can try to write your own connector using Dremio Hub | Dremio