Hi - Just trying to get going with data in a public S3 bucket, but not sure how the config is supposed to work. I entered my credentials, and it says a bucket already exists at the path specified (which is true, and I want to load data from that bucket). If I try creating a new bucket, it fails. What should I be doing?
Hi,
I believe you are running into a known issue. If you try to create a source called Foo and it fails (connection issues, invalid credentials) you won’t be able to use Foo as a Source name after the failure happens. Changing the Source name should allow you to save it successfully.
Doron
The other possibility, given your description, I think you may be explicitly listing buckets for your account in the external buckets section of the configuration. You should only list buckets for external buckets, not ones under your account. Those ones are automatically made available. External buckets are used for things like publically accessible buckets. (For example, you can add samples.dremio.com as an external bucket, and see that in addition to your internal buckets).
Thanks.
Seems like my issue is now more basic - I enter my access ID/secret key, and the primary failure is ‘Failed to create workspaces for buckets in specified account’. I haven’t edited or added any external buckets - i only use the ‘name’ field under the ‘general’ section of the AWS config page. I’m assuming this refers to a bucket that Dremio will create, correct?
When creating an Amazon S3 datasource, you provide a name for the Dremio datasource that you’ll use in building queries later. In the following screen shot, what else did you provide other than Name, AWS Access Key, AWS Access Secret?
hi Venkat,
It looks like you are trying to connect to a public S3 bucket.
Below is a sample of how to access public S3 buckets - you only need to specify a logical name for the S3 data source (eg, “samples” from below) and you will need to specify the public bucket name (eg: samples.dremio.com) in the “External Buckets” section and click Add (note you can add multiple public buckets).
If you are trying to connect to buckets in your protected S3 account, instead of public S3 buckets, then you will need to enter your aws access id and key as Ron shows above in his screenshot. When you click save, Dremio will list all the buckets associated with your AWS account automatically (you do not need to add any bucket names explicitly). You can then start browsing, manipulating and querying data in your S3 buckets.
Let us know if this works for you.
Ok, I think i figured the problem out - I’m using Safari, and for some reason (bug?) the entire sources view in the UI is always blank at any point after the data source addition/dataset addition steps. Everything works in Firefox though - the Dataset/Sources view looks like it’s supposed to.
We actually do not support Safari at this time (documented here) and show a warning when you login, but not when you signup so sorry about that. We hope the next MacOS release’s Safari will be better supported.