# Dremio with Hive and S3

**URL:** <https://community.dremio.com/t/dremio-with-hive-and-s3/11235>\
**Category:** Dremio University\
**Created:** [December 4, 2023, 2:10pm UTC](https://community.dremio.com/t/dremio-with-hive-and-s3/11235 "2023-12-04T14:10:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kleinkauff](https://avatars.discourse-cdn.com/v4/letter/k/a5b964/32.png) [@kleinkauff](https://community.dremio.com/u/kleinkauff)\
**Post date:** [December 4, 2023, 2:10pm UTC](https://community.dremio.com/t/dremio-with-hive-and-s3/11235/1 "2023-12-04T14:10:32Z")

</div>

Hey Dremio folks!

I saw other posts with similar problems but their solutions seem not to work for me.

Recently I started to play with Dremio Community Edition, deployed on Kubernetes.

Build  
24.2.6-202311250456170399-68acbe47  
Edition  
Community Edition  
Build Time  
11/25/2023 05:02:17  
Change Hash  
68acbe47566ebb7424d179776f7dc15fc6b81bdf  
Change Time  
11/25/2023 02:34:32

I’m trying to connect to our Hive Catalog to read Iceberg tables located in S3.  
I’ve added a Hive 3 Source with the following Connection Properties, on Advanced Options:

fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem  
hadoop.fs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider  
fs.s3a.access.key=xxx  
fs.s3a.secret.key=yyy  
dremio.s3.compat=true

The connection seems to work. On /source/ I can see all s3 folders listed. I can even see the metadata for some tables

![image](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/b/b3c33bcf50f29252df211e141d3585e01bf843a5.png)

My problem is when trying to query that table. I’m seeing this error:

AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 490K6Z4K365Q2P6H; S3 Extended Request ID: w0m56lkDQDKlodR4t/W6Svsk0XODaskLKr58e8qtFtkcMTKWKN7Ug1o7bGsmEerigOXj+Zu2knM=; Proxy: null)

Not sure what to do, as Dremio can read the folders and table metadata. Any ideas?

Thank you!!

---

<div class="post-metadata">

**Author:** ![bogdan.coman](https://avatars.discourse-cdn.com/v4/letter/b/f6c823/32.png) [@bogdan.coman](https://community.dremio.com/u/bogdan.coman)\
**Post date:** [December 5, 2023, 4:29am UTC](https://community.dremio.com/t/dremio-with-hive-and-s3/11235/2 "2023-12-05T04:29:14Z")

</div>

Hi @kleinkauff,

Looks like the credentials are invalid, can you double check that’s not the case? Maybe they were valid at the time the source was added.  
Also, you mentioned that the solutions for similar problems didn’t work for you. What were those solutions? If you can cross-reference them, that would be great.

Thanks, Bogdan

---

<div class="post-metadata">

**Author:** ![kleinkauff](https://avatars.discourse-cdn.com/v4/letter/k/a5b964/32.png) [@kleinkauff](https://community.dremio.com/u/kleinkauff)\
**Post date:** [December 5, 2023, 12:26pm UTC](https://community.dremio.com/t/dremio-with-hive-and-s3/11235/3 "2023-12-05T12:26:32Z")

</div>

Hey, thank you so much for trying to help me.

I think the key is ok, this is the actions that are configured.

```
actions = [
  "s3:ListBucket",
  "s3:GetObject*",
  "s3:PutObject*",
  "s3:DeleteObject",
  "s3:PutBucketPublicAccessBlock",
  "s3:CreateBucket",
  "s3:GetBucketAcl",
  "s3:GetBucketLocation"
]

```

Strange thing is if I try other keys that do not have some of the permissions I see a different kind of message, like

 ![image](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/3/3429a1be935b8fefdde8a1a353865e9fbd171282.png)

Some of other suggestions:  
Enable compact mode - [Connect Dremio to Iceberg Hive Catalog located at S3 - #3 by lenoyjacob](https://community.dremio.com/t/connect-dremio-to-iceberg-hive-catalog-located-at-s3/10893/3)

Compat mode in a different setting? - [Connecting S3 bucket via ec2 metadata - #2 by balaji.ramaswamy](https://community.dremio.com/t/connecting-s3-bucket-via-ec2-metadata/7177/2) - Not sure if this could help me

---

<div class="post-metadata">

**Author:** ![lenoyjacob](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/lenoyjacob/32/3474_2.png) [@lenoyjacob](https://community.dremio.com/u/lenoyjacob)\
**Post date:** [December 5, 2023, 4:51pm UTC](https://community.dremio.com/t/dremio-with-hive-and-s3/11235/4 "2023-12-05T16:51:22Z")

</div>

> [@kleinkauff](#):
>
> I think the key is ok, this is the actions that are configured.
> 
> ```auto
> actions = [
> "s3:ListBucket",
> "s3:GetObject*",
> "s3:PutObject*",
> "s3:DeleteObject",
> "s3:PutBucketPublicAccessBlock",
> "s3:CreateBucket",
> "s3:GetBucketAcl",
> "s3:GetBucketLocation"
> ]
> 
> ```

I think you’re [missing](https://docs.dremio.com/current/sonar/data-sources/object/s3/#sample-iam-policy-for-accessing-s3) `s3:ListAllMyBuckets`

---

<div class="post-metadata">

**Author:** ![kleinkauff](https://avatars.discourse-cdn.com/v4/letter/k/a5b964/32.png) [@kleinkauff](https://community.dremio.com/u/kleinkauff)\
**Post date:** [December 7, 2023, 12:57pm UTC](https://community.dremio.com/t/dremio-with-hive-and-s3/11235/5 "2023-12-07T12:57:25Z")

</div>

@lenoyjacob Worked.

@bogdan.coman @lenoyjacob Thank you so much folks!