Dremio Version: 24.0.0
ENV: microk8s
When connecting to S3-compatible storage(Ali OSS which only support virtual hosted accessing) as data source, there is no problem in the webui and coordinator. Files can be loaded in the webui. However, when executing sql, there will be error in executor node as below(“Please use virtual hosted style to access.”).
I have already set fs.s3a.path.style.access to false, but it seems that the config can only affect coordinator node, executor node running sqls still use path style access. Is there any way to solve the problem? Thanks a lot for help!
ERROR:
PERMISSION ERROR: Access was denied by S3
SqlOperatorImpl TABLE_FUNCTION
Location 1:3:4
Fragment 1:0
[Error Id: c4fb5461-0c28-4a56-b94b-166bfe0bb053 on dremio-executor-1.dremio-cluster-pod.dremio.svc.cluster.local:0]
(java.util.concurrent.CompletionException) software.amazon.awssdk.services.s3.model.S3Exception: Please use virtual hosted style to access. (Service: S3, Status Code: 403, Request ID: 64367DF6FF584737324F3D4B, Extended Request ID: null)
software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException():60
software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0():51
java.util.concurrent.CompletableFuture.uniHandle():836
java.util.concurrent.CompletableFuture$UniHandle.tryFire():811