Dremio & Iceberg using ADLS2

1

Hello,
i’m creating some Iceberg tables from Dremio using ADLS2 as catalog. When dremio create Iceberg table, it use the old “wasbs” schema. I want to use abfs, can you provide me some information about how i can settings abfs format?
Thank’s in advice

Emanuele

2

Hi Emanuele,

Greetings!
Thank you for contacting Dremio.
Regarding your query, on checking internally, Dremio currently uses the wasbs:// scheme in the manifest files instead of abfss:// that is Hardcoded and unfortunately I couldn’t find a workaround. However, there is an enhancement in consideration with our engineering team already (Internal ID for record: #DX-66589), although we don’t have an ETA as of now. We will check this further with the internal team on the status. Meanwhile, I would like to confirm if you are using Dremio Cloud edition? If not, then kindly also share your Dremio Software edition version.

Regards,
Shantanu Singh

3

Thank’s for the reply. I’m using Dremio Software in Kubernetes sorry for my mistake. Currently we are using Dremio Community version 24.1. Can you link me the Dremio class with the wasbs hardcode?

Emanuele

4

Thanks for the clarification. Noted that you are running Dremio Software.
While I was initially reviewing this for Enterprise edition. On checking for oss from below git link you can find the Listing of schemes supported in Dremio:

5

Ok thank’s very much for the github’s link.

6

Hi,
Is it planned to be changed in near future?