Unable to select from iceberg table stored in S3 compatible storage with hive catalog

I created a data sou3rce as Hive 3.x metastore to read tables stored in S3 compatible storage with hive catalog in iceberg table format. The following properties are defined in the advanced options:
fs.s3a.access.key= <access.key>
fs.s3a.secret.key=<secret.key>
fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem
fs.s3a.endpoint=
fs.s3a.path.style.access=true
hadoop.fs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider

I can brows the list of databases and tables. But when I try to select from any table I get the following error:
s3a://iceberg-test3/data/metadata/00000-68642976-ae50-4da4-8a64-9ed8a5aa3a58.metadata.json: getFileStatus on s3a://iceberg-test3/data/metadata/00000-68642976-ae50-4da4-8a64-9ed8a5aa3a58.metadata.json: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 0a8b31d2:182c7f08b15:a4bcf:52; S3 Extended Request ID: ; Proxy: null), S3 Extended Request ID: :403 Forbidden.

The access key has full permissions to the bucket.

Any idea what’s happening? Please note that I can select from the same catalog and tables which I created from spark-sql.

  1. Which Dremio version is this?
  2. Which storage are you using?
  3. Which connector are you using? S3 or Hive? If Hive -2 or 3?

There’s s3 request logging that can be enabled via logback.xml that could help us identify which specific request is failing.

  1. Build: 22.1.1-202208230402290397-a7010f28
  2. Storage: Dell ECS
    3 Connector: Hive3

Access key and secret key were switched! I am getting file not found now while the file already exists…

Here is the Exception I am getting now…

2022-12-08 20:26:47,232 [qtp827206475-207] WARN org.apache.iceberg.util.Tasks - Retrying task after failure: Failed to open input stream for file: s3a://iceberg-test3/data/mytable/metadata/00001-ddecd9af-e5cd-404c-905a-dbb5492967e9.metadata.json

org.apache.iceberg.exceptions.NotFoundException: Failed to open input stream for file: s3a://iceberg-test3/data/mytable/metadata/00001-ddecd9af-e5cd-404c-905a-dbb5492967e9.metadata.json

    at org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:177)

    at com.dremio.exec.store.iceberg.DremioInputFile.newStream(DremioInputFile.java:100)

    at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:250)

    at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:245)

    at org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$0(BaseMetastoreTableOperations.java:171)

    at org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$1(BaseMetastoreTableOperations.java:185)

    at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)

    at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)

    at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)

    at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)

    at org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:185)

    at org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:170)

    at org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:165)

    at com.dremio.exec.store.hive.iceberg.IcebergHiveTableOperations.doRefresh(IcebergHiveTableOperations.java:44)

    at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:95)

    at org.apache.iceberg.BaseTable.refresh(BaseTable.java:59)

    at com.dremio.exec.store.hive.metadata.HiveMetadataUtils.getTableMetadataFromIceberg(HiveMetadataUtils.java:561)

    at com.dremio.exec.store.hive.metadata.HiveMetadataUtils.getTableMetadata(HiveMetadataUtils.java:533)

    at com.dremio.exec.store.hive.Hive3StoragePlugin.listPartitionChunks(Hive3StoragePlugin.java:1295)

    at com.dremio.exec.catalog.DatasetSaverImpl.saveUsingV1Flow(DatasetSaverImpl.java:248)

    at com.dremio.exec.catalog.DatasetSaverImpl.save(DatasetSaverImpl.java:121)

    at com.dremio.exec.catalog.DatasetSaverImpl.save(DatasetSaverImpl.java:143)

    at com.dremio.exec.catalog.DatasetManager.getTableFromPlugin(DatasetManager.java:373)

    at com.dremio.exec.catalog.DatasetManager.getTable(DatasetManager.java:215)

    at com.dremio.exec.catalog.CatalogImpl.getTableHelper(CatalogImpl.java:472)

    at com.dremio.exec.catalog.CatalogImpl.getTable(CatalogImpl.java:225)

    at com.dremio.exec.catalog.SourceAccessChecker.lambda$getTable$3(SourceAccessChecker.java:128)

    at com.dremio.exec.catalog.SourceAccessChecker.getIfVisible(SourceAccessChecker.java:97)

    at com.dremio.exec.catalog.SourceAccessChecker.getTable(SourceAccessChecker.java:128)

    at com.dremio.exec.catalog.DelegatingCatalog.getTable(DelegatingCatalog.java:105)

    at com.dremio.exec.catalog.CachingCatalog.getTable(CachingCatalog.java:94)

    at com.dremio.dac.explore.DatasetsResource.getDatasetSummary(DatasetsResource.java:314)

    at com.dremio.dac.explore.DatasetsResource.newUntitled(DatasetsResource.java:152)

    at com.dremio.dac.explore.DatasetsResource.newUntitledFromParent(DatasetsResource.java:247)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:498)

    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)

Caused by: java.io.FileNotFoundException: No such file or directory: s3a://iceberg-test3/data/mytable/metadata/00001-ddecd9af-e5cd-404c-905a-dbb5492967e9.metadata.json

    at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3864)

    at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3686)

    at org.apache.hadoop.fs.s3a.S3AFileSystem.extractOrFetchSimpleFileStatus(S3AFileSystem.java:5399)

    at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:1468)

    at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:1444)

    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:976)

    at org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:175)

    ... 94 common frames omitted

@eelsherif Couple of questions

  • Does this file exists?
    3a://iceberg-test3/data/mytable/metadata/00001-ddecd9af-e5cd-404c-905a-dbb5492967e9.metadata.json

  • If it exists, are you able to read it from the coordinator via command line using the aws S3 commands?