Azure Data Lake V2 Storage and dremio.conf explanation

1

Hi,

can someone put example for dremio.conf
paths: {
…
dist: “dremioAzureStorage://:///<FILE_SYSTEM_NAME>/<ALTERNATIVE_STORAGE_ROOT_DIRECTORY>”
}

What string we need to replace in <FILE_SYSTEM_NAME> and <ALTERNATIVE_STORAGE_ROOT_DIRECTORY>.
Is <FILE_SYSTEM_NAME> mean to be Azure Data Lake Storage Name? Same as ACCOUNT_NAME in core-site.xml?

What string we need to replace in <ALTERNATIVE_STORAGE_ROOT_DIRECTORY>? Do we need to put folders from Azure Data Lake?

Thanks!

2

As an example, if you have an Azure storage account {{myadlsgen2}} and you want Dremio to write results, accelerations etc. to a container called {{dremio_storage}} on that account, then the {{dist}} path would be:
dist: "dremioAzureStorage://:///dremio_storage"

Note: you do not need to specify the account name in this path. Dremio will determine that from your core-site.xml

1 Like