Issue in Connecting S3 as DataSource

1

Hi
I am trying to connect S3 as data source but i am getting this issue
“Failure creating/updating source”
I did the configuration as the https://docs.dremio.com/data-sources/s3.html#advanced-options
Create policy as mention in the instruction.

Thanks

Sanjay

2

HI @Sanjay1625

Can you please provide us with the server.log when the problem happened?

Thanks
@balaji.ramaswamy

3

Hello,

I’m getting the same issue. I set it up with an AWS access and secret key. I validated my credentials via the aws-cli to see if I have access, and I do have access. The dremio-master-0 logs only give me very little (HTTP 400)

10.196.56.58 - - [24/Feb/2020:19:00:38 +0000] “PUT /apiv2/source/Test%20S3?nocache=1582570837873 HTTP/1.1” 400 517 “http://mydremio.com:9047/” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36”

Any help woould be appreciated,

Carlos

4

@carlos.guerra

Would you be able to send us the bucket policy?

https://docs.dremio.com/data-sources/s3.html#sample-iam-policy-for-accessing-s3

5

Hello,

Thank you for replying. The S3 bucket has not Bucket Policy. However the user account accessing the bucket has the following policy below.

Thanks

Carlos

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Action”: [
“s3:PutObject”,
“s3:GetObject”
],
“Resource”: [
“arn:aws:s3:::mybucket-name/dremio/"
],
“Effect”: “Allow”,
“Sid”: “PrefixLevel”
},
{
“Condition”: {
“StringLike”: {
“s3:prefix”: [
“dremio/”,
"dremio/”
]
}
},
“Action”: [
“s3:ListBucket”
],
“Resource”: [
“arn:aws:s3:::mybucket-name”
],
“Effect”: “Allow”,
“Sid”: “BucketLevel”
},
{
“Action”: [
“s3:GetBucketLocation”
],
“Resource”: [
“arn:aws:s3:::mybucket-name”
],
“Effect”: “Allow”,
“Sid”: “BucketLocation”
}
]
}