We currently have a functioning S3 Connection reading JSON files from three buckets, we’ll call:
My test file
We added permissions to read from BucketD, which has identical data to that which is successfully being read in BucketA-C, yet Dremio sees the bucket, let’s you navigate into it and select and save the file as a JSON file, but as soon as it takes you into the Data/SQL view it says
Failure reading JSON file - Unable to find bucket named BucketD, yet if I dismiss the error message, I see displayed in the data view the keys
test, but no data with them.
Dremio Build: 4.0.2-201910020123580864-a98a0b9
Not exactly following this line “We added permissions to read from BucketD, which has identical data to that which is successfully being read in BucketA-C” but in general, we list all buckets for that account and when you try to select if there is no read permissions we fail
Is this your question?
Let me include the policy to clarify:
We added BucketD to the policy. When testing the permissions via the aws cli, we are able to connect, decrypt, and read data from BucketA-D but Dremio acts as though it cannot find BucketD even though it can read the json keys (but not the values) from the file within the BucketD.
@balaji.ramaswamy Does that clarify the question?
Yes got it, can you please send us the job profile
Share Query Profile
Had a similar issue that I got through by readding the S3 connection. (Unable to find bucket named error when querying S3)
Hope that helps.