Cannot create table in Nessie Branch

Dremio Cloud
1

Hi, I have followed the guides to setup a Sonar and Arctic projects in Dremio Cloud, and now I am trying to create a versioned table which fails.

This works:

create branch branch2;
use branch branch2;

This fails:
create table nessie.tab1 as select 1,2,3,4;

with
ContainerNotFoundException: Bucket tab1 not found.

profile zip shows:

"error":"SYSTEM ERROR: ContainerNotFoundException: Bucket tab1 not found\n\nSqlOperatorImpl ICEBERG_MANIFEST_WRITER\nLocation 0:0:6\nSqlOperatorImpl ICEBERG_MANIFEST_WRITER\nLocation 0:0:6\nFragment 0:0\n\n[Error Id: e3a24508-13e9-4a5c-ae7a-2ecf448a72e1 on 172.31.54.171:0]\n\n (org.apache.iceberg.exceptions.RuntimeIOException) Failed to create file: /tab1/metadata/4804481b-fbd6-417f-b78a-95f4e8a66da0.avro\n org.apache.iceberg.hadoop.HadoopOutputFile.createOrOverwrite():87\n org.apache.iceberg.hadoop.DremioOutputFile.createOrOverwrite():48\n org.apache.iceberg.avro.AvroFileAppender.<init>():51\n org.apache.iceberg.avro.Avro$WriteBuilder.build():198\n org.apache.iceberg.ManifestWriter$V1Writer.newAppender():281\n org.apache.iceberg.ManifestWriter.<init>():58\n org.apache.iceberg.ManifestWriter.<init>():34\n org.apache.iceberg.ManifestWriter$V1Writer.<init>():260\n org.apache.iceberg.ManifestFiles.write():117\n org.apache.iceberg.ManifestFiles.write():101\n

and strangely also has another error I did not see in the UI:

  (org.apache.calcite.runtime.CalciteContextException) From line 7, column 3 to line 7, column 6: Object 'json' not found
    sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
    sun.reflect.NativeConstructorAccessorImpl.newInstance():62
    sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
    java.lang.reflect.Constructor.newInstance():423
    org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
    org.apache.calcite.sql.SqlUtil.newContextException():800

542eda47-a1dc-4da3-b7ee-753e3b1e3662.zip (11.5 KB)

2

@tgavin maybe you can help?

3

It looks like the Nessie source was created with a root path of just ‘/’

This needs to be set to ‘/your-bucket-name’ instead.

This is also a UI bug that we should not allow just specifying only ‘/’ for this value. This bug is logged.

4

aws-root-path-default-value-error
aws-root-path-default-value-error652×665 38.9 KB

In the Storage configuration, the default value of “/” needs to be updated to “/your-bucket-name”

Let me know if this resolves the issue. To change this setting you need to create a new Nessie Source.

5

ok, removed and recreated the nessie source just now. I am using /dremio-nessie as AWS Root Path. Thats the same bucket I created for the Artic project, hope thats what this means.

I am running the same statements as above:

create branch branch2;
use branch branch2;

create table nessie2.tab1 as select 1,2,3,4; now fails with

"error":"PERMISSION ERROR: Access was denied by S3\n\nSqlOperatorImpl ICEBERG_MANIFEST_WRITER\nLocation 0:0:6\nSqlOperatorImpl ICEBERG_MANIFEST_WRITER\nLocation 0:0:6\nFragment 0:0\n\n[Error Id: 3288a658-5625-4083-89da-c70a23d52969 on 172.31.58.108:0]\n\n (com.amazonaws.services.s3.model.AmazonS3Exception) Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: QK8K8HZH0W6H3CCY; S3 Extended Request ID: 0Vp3JmnFaK0Q59Wa6ObB5FkNF4rvjVGlTwFXUNWYk0fPGbXd3+ZTw46ey9XIlYFS++HBv+jAoKk=; Proxy: null)\n

0a1a49b3-4ccc-420a-bcf9-37dae7ee371a.zip (12.2 KB)

6

argh! nevermind. Changed AWS Root Path to /dremio-nessie/tables and adding AWS Access Key for authentication fixed it. Now the SQL above works. Magic!

7

Excellent, it is exciting when it works :slight_smile: This feedback is also helpful as we improve our documentation and user experience. Thanks again for posting!