DataBricks Delta Support?

[ig-mageru]

Does anyone know if Dremio will be introducing a DataBricks Delta connector? I see there is one for SnowFlake and Vertica but alot of our data is currently stored in Delta (would love to be able to connect to it and leverage Dremio)

[donghwakim]

I am also interested in this connector as well.

[ddd]

+1

I assume this is not a connector, but rather ability to understand format of data stored as delta or iceberg.

[balaji.ramaswamy]

We are still evaluating the feature. Will keep the thread posted !

[techfoxy]

This should be connector! While Delta format in versioned Parquet, it means Dremio can read the Delta table but will got a consistency issue when the Delta table was updated. Please refer to Hive Delta connector that support to create Hive external table to Delta lake table.

[techfoxy]

Please refer to the Hive Delta Connector and hope a new supported connect for Dremio will be available soon: https://github.com/delta-io/connectors

[ig-mageru]

Thanks for your reply and the information! I have shared this with others that had a similar question here.

Delta lake integration

Out of box Dremio integration with delta lake would be great. We have a use case where all the data ingestion and processing happens through spark and delta lake and we want to leverage interactive query performance of Dremio. So far in all the articles and presentation we have seen dremio and spark data source integration. Other-way integration would be very helpful too.