S3 Resources available, but can't preview or query

Dremio Cloud
1

Hi, I recently set up my AWS stack and connected an S3 bucket to Dremio. I am able to see all files and when clicking the ‘Format File’-button, I am actually able to see the table information properly:

image
image1795×1606 172 KB

However, once I press Save and try to Run or Preview the data, it always fails with the following error:
Failure getting source [Dremio S3 Bucket]

Stack Trace:

(java.util.concurrent.ExecutionException) com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Credential Verification failed.

When I try to query the Sample taxi data, I get this error:
SocketTimeoutException: connect timed out

Stack Trace:

(java.lang.RuntimeException) com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to samples.dremio.com.s3.amazonaws.com:80 [samples.dremio.com.s3.amazonaws.com/52.218.133.201, samples.dremio.com.s3.amazonaws.com/52.92.208.185, samples.dremio.com.s3.amazonaws.com/52.92.128.177, samples.dremio.com.s3.amazonaws.com/52.218.178.107, samples.dremio.com.s3.amazonaws.com/52.92.213.49, samples.dremio.com.s3.amazonaws.com/52.92.210.41, samples.dremio.com.s3.amazonaws.com/52.92.128.137, samples.dremio.com.s3.amazonaws.com/52.92.148.105] failed: connect timed out

What I have tried and checked so far from Googling and looking around on this forum:

  • S3 connection is through Access Key with all the appropriate permissions
  • The private subnet has “Auto-assign public IPv4 address” enabled
  • The VPC has an internet gateway associated with them
  • The private subnet has an internet gateway connected in routing tables
  • I added connection properties ‘dremio.s3.property’ and ‘fs.s3a.endpoint’ pointing to the correct AWS location

At this point I am really not sure how to continue… maybe something very obvious is missing and someone can point me in the right direction? :slight_smile:

2

Could you share exactly how first and second tab of source configuration look like?
Why was there a need to add extra connection properties?

3

Sure thing! I’ll need to split it into two posts as new members can only share 1 image per post.
The reason for the connection properties was because I was simply following all solutions online of people that had the error message, and someone said this fixed it for them.

image
image1825×1629 148 KB

4

image
image1809×1621 167 KB

5

I honestly have no idea why, but today I woke up and it started working out of nowhere.