Connecting S3 bucket via ec2 metadata

Hi Team,
I am having only one S3 bucket and that needs to connect dremio. Due to security concerns i cannot provide access key and secret key. So i have attached one IAM policy without * resource ( Which mentioned below) to Dremio instance and trying to fetch the data. But it is giving the error related to “sts:AssumeRole” and this assumerole policy cannot be given due to security concerns. So Please take this issue and share me the resolution.

Attached IAM Policy:

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Sid”: “Stmt1554423012000”,
“Effect”: “Allow”,
“Action”: [
“s3:GetBucketLocation”,
“s3:ListAllMyBuckets”,
“s3:ListBucket”
],
“Resource”: [
“arn:aws:s3::: my-bucketname”
]
},
{
“Sid”: “Stmt1554423050000”,
“Effect”: “Allow”,
“Action”: [
“s3:GetObject”
],
“Resource”: [
“arn:aws:s3::: my-bucketname*”
]
}
]
}

Error Mentioned by dremio:
"
No AWS Credentials provided by STSCredentialProviderV1 : com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: User: arn:aws:sts::795770151185:assumed-role/bhged-quad-s3-access-new/i-0c7d6945d08245639 is not authorized to perform: sts:AssumeRole on resource: bhged-quad-s3-access-new (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: 31eae4f2-2087-440f-8a75-10c3cca25611) "

Thanks and Regards,
Sarvasiddi Manikanta.

@msarvasiddi

If you edit the S3 source and under advanced options, enable compatibility mode, does it help?

We have enabled the compatibility mode and attached all the details in the required spaces. But getting the same error related to Sts:AssumeRole

Please find the exact error below

“java.nio.file.AccessDeniedException: gcp-s3-migration: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials provided by STSCredentialProviderV1 : com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: User: arn:aws:sts::795770151185:assumed-role/bhged-quad-s3-access/i-0c7d6945d08245639 is not authorized to perform: sts:AssumeRole on resource: bhged-quad-s3-access (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: 85341c17-89cc-401d-8f64-b9353cc40b79)”

Please check this

@msarvasiddi

Can you also add the below to core-site.xml and restart Dremio cluster, retry?

<property>>
<name>dremio.s3.compat</name>
<description>Compatiable mode, not calling AWS STS</description>
<value>true</value>
</property>