Connecting Dremio to Storage Grid - s3

1

Hi.

I’m trying to connect dremio to s3 StorageGrid via a kubernetes deployment. I have edited the fs.s3a.endpoint in core-site.xml to point to our inhouse endpoint. I am however getting a User exception when I try to build the data source with the following timeout exception:

com.dremio.common.exceptions.UserException: Failure while configuring source []
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-3.1.9-201904051346520183-a35b753.jar:3.1.9-201904051346520183-a35b753]
at com.dremio.exec.catalog.CatalogServiceImpl.createOrUpdateSource(CatalogServiceImpl.java:639) [dremio-sabot-kernel-3.1.9-201904051346520183-a35b753.jar:3.1.9-201904051346520183-a35b753]
at com.dremio.exec.catalog.CatalogServiceImpl.createSource(CatalogServiceImpl.java:378) [dremio-sabot-kernel-3.1.9-201904051346520183-a35b753.jar:3.1.9-201904051346520183-a35b753]
at com.dremio.exec.catalog.CatalogServiceImpl.access$600(CatalogServiceImpl.java:102) [dremio-sabot-kernel-3.1.9-201904051346520183-a35b753.jar:3.1.9-201904051346520183-a35b753]
at com.dremio.exec.catalog.CatalogServiceImpl$SourceModifier.createSource(CatalogServiceImpl.java:972) [dremio-sabot-kernel-3.1.9-201904051346520183-a35b753.jar:3.1.9-201904051346520183-a35b753]
at com.dremio.exec.catalog.CatalogImpl.createSource(CatalogImpl.java:519) [dremio-sabot-kernel-3.1.9-201904051346520183-a35b753.jar:3.1.9-201904051346520183-a35b753]
at com.dremio.exec.catalog.DelegatingCatalog.createSource(DelegatingCatalog.java:189) [dremio-sabot-kernel-3.1.9-201904051346520183-a35b753.jar:3.1.9-201904051346520183-a35b753]

Any help with this would be appreciated.

Thanks

2

Hello @gaurisrinivas,

Are you trying to configure S3 as a source in Dremio? This is what the error stack implies.

If that’s the case, you do not need to add anything to core-site.xml (you actually don’t require that configuration file at all). You can do all the configuration via the UI.