ADLS Gen2 storage access using Oauth

1

Hi Team,

I am new to Dermio and I am currently doing POC to evaluate dremio for our usecase. I have installed dremio in AKS and trying to create azure storage source. Unfortunately I am not able succeed to add this source.

I have service principles only to specific directory in my container(e g storage_accout/container/directory/subdir). My organization restricted to access only though OAUTH. I tried to configure new data lake source(azure storage) with all combinations , but nothing worked out.

My question are ,

  1. Does dremio required access at container level? If not please help me the detailed steps to configure the azure source with OAUTH. I have found the document with only basic settings, I did not find help how to set the advanced options in my case.
  2. How to enable debug options? I am getting access denied error when I check master pod logs, but I did not find much info with that error log.
  3. If my POC goes well and satisfied with the results, how to obtain enterprise edition.
    4 Difference between installing Dermio in my AKS vs Install through market place. Advantages and disadvantages. It would be great if I can get the pricing details.

Regards,
Ramesh

2

@rammi16 Maybe give the source path you have access in the advanced options root folder?

Screen Shot 2021-08-15 at 11.52.10 PM
Screen Shot 2021-08-15 at 11.52.10 PM2648×572 54.5 KB

3

Thank you! Sorry for the late response. It was a permission issue. Dremio expecting to have list permissions from adls file system to till the sub directory.

Interesting thing is, it doesn’t require list permissions till sub directory to access from spark.

Now I am trying to connect dremio from superset, I can able to establish the connection, but unable to create the dataset.

Does anyone succeed to create the dremio dataset in superset?

Appreciated if any provide the list of steps. I am really stuck here. I have posted same question in superset community, no response yet

Regards,
Ramesh

4

@rammi16 I am checking internally on superset and will get back to you

5

Thank you Balaji! I am able to create the datasets and created the reports as well.

Regards,
Ramesh