Support for Azure Blob Storage

#1

Hi,
I’ve discovered Dremio recently and I think that’s a really amazing tool. I’d like to use it with Azure Blob Storage, but as far as I know, there’s no plugin for that. I’m trying to implement one, but I can´t find a developer guide. Could you tell me where can I find information related to plugin architecture?

0 Likes

#2

We recently added support for ADLS: http://docs.dremio.com/data-sources/azure-data-lake-store.html

Does that help? We’re working on an SDK for users to add their own sources, but that’s not yet available.

0 Likes

#3

Hey @Andy, as Kelly mentioned, we’ve recently added support for Azure Data Lake Store. We have plans to add support for Azure Blob Storage in late Q1/mid-Q2.

0 Likes

#4

Hi everyone,

Thank you for the answers. I’m testing the use of the virtual file system adapter for Azure Blob storage (https://github.com/Azure/azure-storage-fuse) to overcoming the issue, but the latency is causing that the virtual machine crashed when Dremio converts the directory in a Parquet table.
If I copy the whole directory locally and configure a source as NAS, Dremio is able to perform the conversion. Do you have any suggestions?

0 Likes

#5

Hello,
Is there an update for this issue ?
In an earlier post it was said that it was a feature planned for Q1/mid-Q2 2018.

0 Likes

#6

Hello @kelly,
Any update on the SDK? Is it available yet?

1 Like

#7

@can Any update on the Blob Storage?

0 Likes