Dremio add S3 data source from different aws account

1

just wondering how to add a S3 data source cross aws accounts?

due to the company’s policy, i have to use an IAM role instead of access key and secret to grant the permission to s3 buckets, i have successfully created an IAM role and attached to the EC2 instances where the dremio is deployed. I can successfully see all buckets associated with the IAM roles.

but i also want to access to another bucket from another aws account. i have updated the bucket policies so that the dremio cluster can successfully download the content of the bucket (e.g. aws s3 cp s3://bucket-from-another-aws-account local), but the dremio dashboard could show this bucket properly as an external bucket. I got 403 error Access denied.

Any idea how to add a S3 bucket as a data source from a different aws account? thanks

1 Like
2

I believe to do this you’ll have to create a new source with the second set of credentials.

You can query both datasets equally, so this shouldn’t be limiting but let us know if you run into anything.

3

Hi Kelly, what did you mean with the second set of credentials? does it mean access key and secret? Is it possible to use IAM-role to control the accessibility? as I mentioned above, i have updated the bucket policies so that the dremio server can successfully download the content of the bucket from another aws account, but how to let the dremio dashboard show this bucket as a new source?

4

Yes you can use IAM:

https://docs.dremio.com/data-sources/s3.html

Set up one source per credential.