I’m using Hive on top of S3 as my physical datasource on Dremio for evaluation purpose. I’ve a few queries around it:
- Do all Hive tables be created with s3a filesystem? I get the following error when i try to connect to an external hive table created on top of s3 filesystem -
Is there a way around for dremio read s3 or s3n filesystems as well?
- How is the access control on Hive tables managed?
I’ve a few tables created in hive with s3a filesystem and I was able to connect to them and query them. But i’m getting the following error right now -
2018-12-31 02:40:58,971 [qtp382934088-1539] INFO c.d.e.catalog.ManagedStoragePlugin - User Error Occurred [ErrorId: d3c33d5b-0b9d-4787-ba1a-15b99ca8de30] com.dremio.common.exceptions.UserException: Access denied reading dataset "connstring".schema.tablename.
There is no change in the instance profile that grants the s3 acess. There is no change in the security group to access hive thrift server. Infact i even tried removing the source and adding it back. It retrieves the tables when the new connection is added; but still throws the same Access Denied error when querying.