I don’t see an option for Elasticsearch. It is assumed that we need to set up a pipeline from Elasticsearch to S3 before we can consume the data in Dremio?
I also tried looking for an Elasticsearch option in the data lakes/sources when using the Dremio Software (AWS Marketplace offering) and it didn’t seem to be present. It did have more options than Dremio Cloud, though.
Hi @redgeoff, you’re correct that Dremio Software (self-managed) supports Elasticsearch as an external data source. However, Dremio Cloud currently doesn’t support Elasticsearch as a data source. Our external sources for Dremio Cloud are listed in the Dremio Cloud docs. We’ll look into clarifying the distinction between our Software and Cloud docs in future.
If you’re using Dremio Software via AWS Marketplace, you can add Elasticsearch as an external data source by clicking “External Sources” in the Dremio UI. Can you please let us know if that works?
For Dremio Cloud, not sure what your data pipeline to get data into Elasticsearch is but Dremio could definitely access it if that Elasticsearch data somehow made its way into S3.
Hi @redgeoff, my apologies for the miscommunication here. Elasticsearch isn’t available as an external source for AWS Edition, but available in other Dremio Software deployment options. We’ll make this clearer in the Elasticsearch source docs.
