Error configuring Storage s3

Hello! I’m configuring the storage s3 in Dremio to store Dremio data (Reflections, Uploads, etc …), but I’m having problems. I did the configuration following the doc, but it doesn’t work. Here are the settings:


<?xml version="1.0"?>
    <description>The FileSystem implementation. Must be set to</description>
    <description>AWS access key ID.</description>
    <description>AWS secret key.</description>
    <description>The credential provider type.</description>


paths: {
  # the local path for dremio to store data.
  local: "/var/lib/dremio"

  # the distributed path Dremio data including job results, downloads, uploads, etc
  #dist: "pdfs://"${paths.local}"/pdfs"
  dist: "s3a://dremio-reflections/reflections"

services: {
  coordinator.enabled: true,
  coordinator.master.enabled: true,
  executor.enabled: true

Caused by: java.nio.file.AccessDeniedException: dremio-reflections: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials provided by InstanceProfileCredentialsProvider : com.amazonaws.SdkClientException: The requested metadata is not found

you are missing / in s3a:///dremio-reflections/reflections

according documentation must be "dremioS3:///<bucket_name>/<folder1>/<folder2>"

if above not work, please consider this:
if you use InstanceProfileCredentialsProvider you can’t use MY-ACCESS-KEY and MY-SECRET-KEY you must use org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider

1 Like

Sorry, lack of attention on my part. It worked here, thanks @dacopan!