Storing reflection data in S3

1

I would like to store reflection data in S3, but cannot find any documentation of how to configure the connection. In dremio.conf I imagine I will need to put the path to the s3 bucket in the paths.dist field, but how do I specify the S3 access credentials?

2

You need to put your s3 credentials into core-site.xml

<property>
  <name>fs.s3a.access.key</name>
  <value>ACCESS-KEY</value>
</property>

<property>
  <name>fs.s3a.secret.key</name>
  <value>SECRET-KEY</value>
</property>

and have it sourced on dremio classpath (for example copy it to dremio conf directory)

3

Perfect, thanks for the reply. Is this documented somewhere?

4

Not at this time, but we will be adding it shortly. Thanks for pointing this out!