ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found

Pavlo · October 28, 2024, 9:43pm

Hello,
can you please help with the following issue?

When querying table from AWS Glue catalog Dremio fails with:

ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found

I’m following the steps from the blog post: https://www.dremio.com/blog/bi-dashboards-with-apache-iceberg-using-aws-glue-and-apache-superset/#h-connecting-your-aws-glue-catalog-to-dremio

AWS Glue catalog was added successfully. AWS keys were provided for authentication.
Iceberg Tables are listed.
But when the SQL is executed the job fails with the ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found error.

Here is the snippet from logs:

dremio-oss  | 2024-10-28 21:29:26,264 [FABRIC-rpc-event-queue] INFO  c.d.sabot.exec.FragmentExecutors - Received remote fragment start
instruction for 18dffdc8-f27b-812b-155f-fe7c8acc0800:0:0 with assigned weight 1 and scheduling weight 1
dremio-oss  | 2024-10-28 21:29:26,799 [e12 - 18dffdc8-f27b-812b-155f-fe7c8acc0800:frag:0:0] ERROR c.d.plugins.s3.store.S3FileSystem - E
rror while listing S3 buckets
dremio-oss  | java.util.concurrent.ExecutionException: java.io.IOException: From option fs.s3a.aws.credentials.provider java.lang.Class
NotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:594)
dremio-oss  |   at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:553)
dremio-oss  |   at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:110)
dremio-oss  |   at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:247)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2349)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2189)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2079)
dremio-oss  |   at com.google.common.cache.LocalCache.get(LocalCache.java:4017)
dremio-oss  |   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
dremio-oss  |   at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem.getS3V1Client(S3FileSystem.java:829)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem.getContainerCreators(S3FileSystem.java:289)
dremio-oss  |   at com.dremio.plugins.util.ContainerFileSystem.refreshFileSystems(ContainerFileSystem.java:108)
dremio-oss  |   at com.dremio.plugins.util.ContainerFileSystem.initialize(ContainerFileSystem.java:185)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3611)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:554)
dremio-oss  |   at com.dremio.exec.hadoop.HadoopFileSystem.get(HadoopFileSystem.java:123)
dremio-oss  |   at com.dremio.exec.store.hive.exec.DremioFileSystem.initialize(DremioFileSystem.java:114)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3611)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:554)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader$1.lambda$load$0(HadoopFsCacheWrapperPluginClas
sLoader.java:76)
dremio-oss  |   at java.base/java.security.AccessController.doPrivileged(Native Method)
dremio-oss  |   at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
dremio-oss  |   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader$1.load(HadoopFsCacheWrapperPluginClassLoader.j
ava:80)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader$1.load(HadoopFsCacheWrapperPluginClassLoader.j
ava:60)
dremio-oss  |   at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2189)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2079)
dremio-oss  |   at com.google.common.cache.LocalCache.get(LocalCache.java:4017)
dremio-oss  |   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
dremio-oss  |   at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader.lambda$getHadoopFsSupplierPluginClassLoader$0(
HadoopFsCacheWrapperPluginClassLoader.java:96)
dremio-oss  |   at com.dremio.exec.store.hive.Hive3StoragePlugin.createFileSystem(Hive3StoragePlugin.java:419)
dremio-oss  |   at com.dremio.exec.store.hive.Hive3StoragePlugin.createFSWithAsyncOptions(Hive3StoragePlugin.java:456)
dremio-oss  |   at com.dremio.plugins.awsglue.store.AWSGlueStoragePlugin.createFSWithAsyncOptions(AWSGlueStoragePlugin.java:380)
dremio-oss  |   at com.dremio.exec.store.iceberg.IcebergManifestListScanCreator.create(IcebergManifestListScanCreator.java:47)
dremio-oss  |   at com.dremio.exec.store.iceberg.IcebergManifestListScanCreator.create(IcebergManifestListScanCreator.java:34)
dremio-oss  |   at com.dremio.sabot.driver.OperatorCreatorRegistry.getProducerOperator(OperatorCreatorRegistry.java:132)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitSubScan(PipelineCreator.java:241)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitSubScan(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractSubScan.accept(AbstractSubScan.java:81)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:289)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.config.AbstractTableFunctionPOP.accept(AbstractTableFunctionPOP.java:46)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:289)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.config.AbstractTableFunctionPOP.accept(AbstractTableFunctionPOP.java:46)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitLimit(AbstractPhysicalVisitor.java:99)
dremio-oss  |   at com.dremio.exec.physical.config.Limit.accept(Limit.java:60)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitLimit(AbstractPhysicalVisitor.java:99)
dremio-oss  |   at com.dremio.exec.physical.config.Limit.accept(Limit.java:60)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitProject(AbstractPhysicalVisitor.java:84)
dremio-oss  |   at com.dremio.exec.physical.config.Project.accept(Project.java:54)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitProject(AbstractPhysicalVisitor.java:84)
dremio-oss  |   at com.dremio.exec.physical.config.Project.accept(Project.java:54)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitWriter(AbstractPhysicalVisitor.java:69)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractWriter.accept(AbstractWriter.java:30)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitWriterCommiter(AbstractPhysicalVisitor.java:59)
dremio-oss  |   at com.dremio.exec.physical.config.WriterCommitterPOP.accept(WriterCommitterPOP.java:119)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitProject(AbstractPhysicalVisitor.java:84)
dremio-oss  |   at com.dremio.exec.physical.config.Project.accept(Project.java:54)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitScreen(PipelineCreator.java:270)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitScreen(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.config.Screen.accept(Screen.java:60)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator.get(PipelineCreator.java:109)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator.get(PipelineCreator.java:103)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor.setupExecution(FragmentExecutor.java:724)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor.run(FragmentExecutor.java:514)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run(FragmentExecutor.java:1234)
dremio-oss  |   at com.dremio.sabot.task.AsyncTaskWrapper.run(AsyncTaskWrapper.java:130)
dremio-oss  |   at com.dremio.sabot.task.slicing.SlicingThread.mainExecutionLoop(SlicingThread.java:279)
dremio-oss  |   at com.dremio.sabot.task.slicing.SlicingThread.run(SlicingThread.java:186)
dremio-oss  | Caused by: java.io.IOException: From option fs.s3a.aws.credentials.provider java.lang.ClassNotFoundException: Class com.d
remio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.loadAWSProviderClasses(S3AUtils.java:611)
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.buildAWSProviderList(S3AUtils.java:634)
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.createAWSCredentialProviderSet(S3AUtils.java:585)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem.createS3V1Client(S3FileSystem.java:223)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem$3.load(S3FileSystem.java:191)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem$3.load(S3FileSystem.java:187)
dremio-oss  |   at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
dremio-oss  |   ... 83 common frames omitted
dremio-oss  | Caused by: java.lang.ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2597)
dremio-oss  |   at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2668)
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.loadAWSProviderClasses(S3AUtils.java:608)
dremio-oss  |   ... 90 common frames omitted
dremio-oss  | 2024-10-28 21:29:26,804 [e12 - 18dffdc8-f27b-812b-155f-fe7c8acc0800:frag:0:0] ERROR c.d.s.exec.fragment.FragmentExecutor
- ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  | com.dremio.common.exceptions.UserException: ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsPr
ovider not found
dremio-oss  |   at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:984)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor.retire(FragmentExecutor.java:877)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor.finishRun(FragmentExecutor.java:768)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor.run(FragmentExecutor.java:604)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run(FragmentExecutor.java:1234)
dremio-oss  |   at com.dremio.sabot.task.AsyncTaskWrapper.run(AsyncTaskWrapper.java:130)
dremio-oss  |   at com.dremio.sabot.task.slicing.SlicingThread.mainExecutionLoop(SlicingThread.java:279)
dremio-oss  |   at com.dremio.sabot.task.slicing.SlicingThread.run(SlicingThread.java:186)
dremio-oss  | Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: java.io.IOException
: java.util.concurrent.ExecutionException: java.io.IOException: From option fs.s3a.aws.credentials.provider java.lang.ClassNotFoundExce
ption: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2085)
dremio-oss  |   at com.google.common.cache.LocalCache.get(LocalCache.java:4017)
dremio-oss  |   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
dremio-oss  |   at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader.lambda$getHadoopFsSupplierPluginClassLoader$0(
HadoopFsCacheWrapperPluginClassLoader.java:96)
dremio-oss  |   at com.dremio.exec.store.hive.Hive3StoragePlugin.createFileSystem(Hive3StoragePlugin.java:419)
dremio-oss  |   at com.dremio.exec.store.hive.Hive3StoragePlugin.createFSWithAsyncOptions(Hive3StoragePlugin.java:456)
dremio-oss  |   at com.dremio.plugins.awsglue.store.AWSGlueStoragePlugin.createFSWithAsyncOptions(AWSGlueStoragePlugin.java:380)
dremio-oss  |   at com.dremio.exec.store.iceberg.IcebergManifestListScanCreator.create(IcebergManifestListScanCreator.java:47)
dremio-oss  |   at com.dremio.exec.store.iceberg.IcebergManifestListScanCreator.create(IcebergManifestListScanCreator.java:34)
dremio-oss  |   at com.dremio.sabot.driver.OperatorCreatorRegistry.getProducerOperator(OperatorCreatorRegistry.java:132)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitSubScan(PipelineCreator.java:241)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitSubScan(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractSubScan.accept(AbstractSubScan.java:81)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:289)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.config.AbstractTableFunctionPOP.accept(AbstractTableFunctionPOP.java:46)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:289)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitTableFunction(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.config.AbstractTableFunctionPOP.accept(AbstractTableFunctionPOP.java:46)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitLimit(AbstractPhysicalVisitor.java:99)
dremio-oss  |   at com.dremio.exec.physical.config.Limit.accept(Limit.java:60)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitLimit(AbstractPhysicalVisitor.java:99)
dremio-oss  |   at com.dremio.exec.physical.config.Limit.accept(Limit.java:60)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitProject(AbstractPhysicalVisitor.java:84)
dremio-oss  |   at com.dremio.exec.physical.config.Project.accept(Project.java:54)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitProject(AbstractPhysicalVisitor.java:84)
dremio-oss  |   at com.dremio.exec.physical.config.Project.accept(Project.java:54)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitWriter(AbstractPhysicalVisitor.java:69)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractWriter.accept(AbstractWriter.java:30)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitWriterCommiter(AbstractPhysicalVisitor.java:59)
dremio-oss  |   at com.dremio.exec.physical.config.WriterCommitterPOP.accept(WriterCommitterPOP.java:119)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:308)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitOp(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.base.AbstractPhysicalVisitor.visitProject(AbstractPhysicalVisitor.java:84)
dremio-oss  |   at com.dremio.exec.physical.config.Project.accept(Project.java:54)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitScreen(PipelineCreator.java:270)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator$CreatorVisitor.visitScreen(PipelineCreator.java:123)
dremio-oss  |   at com.dremio.exec.physical.config.Screen.accept(Screen.java:60)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator.get(PipelineCreator.java:109)
dremio-oss  |   at com.dremio.sabot.driver.PipelineCreator.get(PipelineCreator.java:103)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor.setupExecution(FragmentExecutor.java:724)
dremio-oss  |   at com.dremio.sabot.exec.fragment.FragmentExecutor.run(FragmentExecutor.java:514)
dremio-oss  |   ... 4 common frames omitted
dremio-oss  | Caused by: java.lang.RuntimeException: java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException:
 From option fs.s3a.aws.credentials.provider java.lang.ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvi
der not found
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader$1.load(HadoopFsCacheWrapperPluginClassLoader.j
ava:82)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader$1.load(HadoopFsCacheWrapperPluginClassLoader.j
ava:60)
dremio-oss  |   at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2189)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2079)
dremio-oss  |   ... 58 common frames omitted
dremio-oss  | Caused by: java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: From option fs.s3a.aws.cred
entials.provider java.lang.ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem.getContainerCreators(S3FileSystem.java:311)
dremio-oss  |   at com.dremio.plugins.util.ContainerFileSystem.refreshFileSystems(ContainerFileSystem.java:108)
dremio-oss  |   at com.dremio.plugins.util.ContainerFileSystem.initialize(ContainerFileSystem.java:185)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3611)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:554)
dremio-oss  |   at com.dremio.exec.hadoop.HadoopFileSystem.get(HadoopFileSystem.java:123)
dremio-oss  |   at com.dremio.exec.store.hive.exec.DremioFileSystem.initialize(DremioFileSystem.java:114)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3611)
dremio-oss  |   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:554)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader$1.lambda$load$0(HadoopFsCacheWrapperPluginClas
sLoader.java:76)
dremio-oss  |   at java.base/java.security.AccessController.doPrivileged(Native Method)
dremio-oss  |   at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
dremio-oss  |   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
dremio-oss  |   at com.dremio.exec.store.hive.exec.HadoopFsCacheWrapperPluginClassLoader$1.load(HadoopFsCacheWrapperPluginClassLoader.j
ava:80)
dremio-oss  |   ... 63 common frames omitted
dremio-oss  | Caused by: java.util.concurrent.ExecutionException: java.io.IOException: From option fs.s3a.aws.credentials.provider java
.lang.ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:594)
dremio-oss  |   at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:553)
dremio-oss  |   at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:110)
dremio-oss  |   at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:247)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2349)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2189)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2079)
dremio-oss  |   at com.google.common.cache.LocalCache.get(LocalCache.java:4017)
dremio-oss  |   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
dremio-oss  |   at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem.getS3V1Client(S3FileSystem.java:829)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem.getContainerCreators(S3FileSystem.java:289)
dremio-oss  |   ... 76 common frames omitted
dremio-oss  | Caused by: java.io.IOException: From option fs.s3a.aws.credentials.provider java.lang.ClassNotFoundException: Class com.d
remio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.loadAWSProviderClasses(S3AUtils.java:611)
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.buildAWSProviderList(S3AUtils.java:634)
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.createAWSCredentialProviderSet(S3AUtils.java:585)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem.createS3V1Client(S3FileSystem.java:223)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem$3.load(S3FileSystem.java:191)
dremio-oss  |   at com.dremio.plugins.s3.store.S3FileSystem$3.load(S3FileSystem.java:187)
dremio-oss  |   at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
dremio-oss  |   at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
dremio-oss  |   ... 83 common frames omitted
dremio-oss  | Caused by: java.lang.ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found
dremio-oss  |   at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2597)
dremio-oss  |   at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2668)
dremio-oss  |   at org.apache.hadoop.fs.s3a.S3AUtils.loadAWSProviderClasses(S3AUtils.java:608)
dremio-oss  |   ... 90 common frames omitted
dremio-oss  | 2024-10-28 21:29:26,820 [Fabric-RPC-Offload12] INFO  c.d.exec.maestro.FragmentTracker - Fragment 18dffdc8-f27b-812b-155f-
fe7c8acc0800:0:0 failed, cancelling remaining fragments.
dremio-oss  | 2024-10-28 21:29:26,823 [Fabric-RPC-Offload12] INFO  c.d.exec.work.foreman.AttemptManager - 18dffdc8-f27b-812b-155f-fe7c8
acc0800: State change requested RUNNING --> FAILED, Exception com.dremio.common.exceptions.UserRemoteException: SYSTEM ERROR: ClassNotF
oundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found

balaji.ramaswamy · November 1, 2024, 12:13am

@Pavlo To just rule out that OSS build is not the issue, can you please download the community version and try the same?

neriberto · November 1, 2024, 7:26pm

I’m encountering the same error. I’m running Docker on an EC2 instance to test, and I’ve tried both dremio/dremio-oss:25.1.1 and dremio/dremio-oss:latest

Reading directly from S3 works, but accessing data from the Glue Catalog does not.

balaji.ramaswamy · November 7, 2024, 6:29am

@neriberto Does it work on Community edition instead of OSS? Reason I am asking is that I see a Class com.d remio.exec.store.hive.GlueAWSCredentialsProvider not found

sheinbergon · January 13, 2025, 3:57pm

@balaji.ramaswamy this is still broken On dremio 25.2.0. Any plans on fixing this issue? How are we supposed to connect to glue in the OSS community edition?

sheinbergon · January 13, 2025, 4:22pm

@balaji.ramaswamy your build is completely broken. GlueAWSCredentialsProvider exists, but it’s path has completely changed, probably as part of your plugin refactoring. You just need to fix the OSS repo to relate to the updated path.

How is this stuff still happening with Dremio after being on the market for so long?

balaji.ramaswamy · January 13, 2025, 11:43pm

@sheinbergon Checking on this internally

sheinbergon · January 14, 2025, 8:50am

Update - this is only happening when using secret access key authentication. When using IAM role, it’s working as expected. The issue probably resides in this line

balaji.ramaswamy · January 14, 2025, 3:51pm

@sheinbergon Yes, Engineering will fix it in the next release

Topic		Replies	Views
Unable to connect to Glue Data Catalog on Dremio 22.0.0	0	1136	July 14, 2022
Problems with connection to AWS Glue Catalog Iceberg table Dremio University	4	1709	September 13, 2022
Unable to access iceberg data created through AWS glue and stored in s3 Dremio Cloud	7	921	September 14, 2023
Connect Dremio to Iceberg Hive Catalog located at S3 Dremio University	2	1156	August 15, 2023
Iceberg Dataset not found	3	1374	August 17, 2022

ClassNotFoundException: Class com.dremio.exec.store.hive.GlueAWSCredentialsProvider not found

Related topics