# Connection String Options for Hive

**URL:** <https://community.dremio.com/t/connection-string-options-for-hive/167>\
**Category:** Uncategorized\
**Created:** [August 7, 2017, 10:31pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167 "2017-08-07T22:31:36Z")\
**Posts on this page:** 19\
**Page:** 2

<div class="post-metadata">

**Author:** ![maver1ck](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@maver1ck](https://community.dremio.com/u/maver1ck)\
**Post date:** [September 22, 2017, 2:40pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/21 "2017-09-22T14:40:20Z")

</div>

@laurent  
I will try the second solution.  
Setting master address is not a solution, because with HA master can change.

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 5, 2017, 10:52am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/22 "2017-12-05T10:52:36Z")

</div>

how to connect to a hive with kerberos enabled? besides filling the “kerberos princple”, is it required to fill the below properties (referring from presto, no idea what key should be specified here)

- hive.metastore.service.principal
- hive.metastore.client.keytab

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 5, 2017, 11:31am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/23 "2017-12-05T11:31:16Z")

</div>

i have installed dremio 1.3 from the tarball on hdp2.6, and added below config to the dremo.conf

- [services.kerberos.principal="ubuntu@DEV.COM](mailto:services.kerberos.principal=%22ubuntu@DEV.COM)"
- services.kerberos.keytab.file.path=/etc/security/keytabs/ubuntu.headless.keytab

and fill the hive connection proptery as:  
 ![dremio-hive](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/b35355c551e705560b184f5aa83e5fa8a6ad2a8c.jpg)

When clck the “hive-dev” source, the dremo server.log has error:  
2017-12-05 19:24:57,895 [qtp826104455-198] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null  
org.apache.thrift.transport.TTransportException: null  
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ~[libthrift-0.9.2.jar:0.9.2]  
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) ~[libthrift-0.9.2.jar:0.9.2]  
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[libthrift-0.9.2.jar:0.9.2]  
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[libthrift-0.9.2.jar:0.9.2]  
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[libthrift-0.9.2.jar:0.9.2]  
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) ~[libthrift-0.9.2.jar:0.9.2]  
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv\_get\_all\_databases(ThriftHiveMetastore.java:739) ~[hive-metastore-1.2.1.jar:1.2.1]  
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get\_all\_databases(ThriftHiveMetastore.java:727) ~[hive-metastore-1.2.1.jar:1.2.1]  
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1031) ~[hive-metastore-1.2.1.jar:1.2.1]  
at com.dremio.exec.store.hive.HiveClient$2.run(HiveClient.java:154) [dremio-hive-plugin-1.3.0-201711211846350824-31f8d91.jar:1.3.0-201711211846350824-31f8d91]

Can anyone has any advice on connecting to a hive with kerberos enabled?

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 5, 2017, 4:34pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/24 "2017-12-05T16:34:25Z")

</div>

my dremio.conf file is:

master: {  
service  
name: [dev.com](http://dev.com),  
port: 45678  
}

paths: {  
local: ${DREMIO\_HOME}"/data"  
}

services: {  
coordinator.enabled: true,  
executor.enabled: true,  
kerberos.principal: "ubuntu@DEV.COM",  
kerberos.keytab.file.path: /etc/security/keytabs/ubuntu.headless.keytab  
}

zookeeper: "[dev.com:2181](http://dev.com:2181)"  
services: {  
coordinator.embedded\_master\_zk.enabled:false  
}

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [December 5, 2017, 4:56pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/25 "2017-12-05T16:56:40Z")

</div>

Did you click on “Enable SASL” to enable it? The box shows as empty on your screenshot.

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 6, 2017, 4:26am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/26 "2017-12-06T04:26:11Z")

</div>

As advised, SASLis enabled, but showing below error when clicking “save”

 ![dremio-hive-error](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/7c2c572363b3d09f3f20159bf2bf11baff91fb87.PNG)

the server.log showed:  
Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to meta store using any of the URIs provided. Most  
recent failure: org.apache.thrift.transport.TTransportException: GSS initiate failed  
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)  
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)  
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)  
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)  
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)  
at java.security.AccessController.doPrivileged(Native Method)

The dremio is running with ubuntu accout, it can connect to hive via beeline.

is possbile something wroing with my dremio.conf ?

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [December 6, 2017, 6:05am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/27 "2017-12-06T06:05:04Z")

</div>

I don’t think it is related to dremio.conf. You do have Hive Metastore running and can access it via hive shell (I know you mentioned beeline, but just to make sure) using kerberos creds you added when configuring Hive in Dremio. Can you also see if Hive Metastore logs show anything.

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 6, 2017, 10:40am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/28 "2017-12-06T10:40:29Z")

</div>

Logging initialized using configuration in file:/etc/hive/2.6.2.0-205/0/hive-log4j.properties  
hive\> use dev;  
OK  
Time taken: 6.592 seconds  
hive\> select \* from mytable;  
OK  
1 2 NULL

Hive metastore log showed:

Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed  
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199)  
at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)  
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)  
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)  
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)  
… 10 more

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 6, 2017, 11:14am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/29 "2017-12-06T11:14:14Z")

</div>

have tried the kerberos testing tool [https://github.com/Teradata/kylo/tree/master/core/kerberos/kerberos-test-client](https://github.com/Teradata/kylo/tree/master/core/kerberos/kerberos-test-client), it’s successful

File Count: 12  
Generating Kerberos ticket for principal: [ubuntu@DEV.COM](mailto:ubuntu@DEV.COM) at key tab location: /etc/security/keytabs/ubuntu.headless.keytab

Sucessfully got a kerberos ticket in the JVM

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [December 6, 2017, 6:43pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/30 "2017-12-06T18:43:34Z")

</div>

What user do you start dremio with - same user you have kerberos principal for?  
If you start dremio with “dremio” user but try to use kerberos principal for a different one it is not going to work.  
You can also try to enable Kerberos debug on dremio to see Kerberos messages:  
in dremio-env uncomment  
#DREMIO\_JAVA\_EXTRA\_OPTS=  
and set it to:

DREMIO\_JAVA\_EXTRA\_OPTS=-Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true -Djavax.net.debug=all

restart dremio and see what additional info you will get when trying to setup hive source

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 7, 2017, 8:50am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/31 "2017-12-07T08:50:46Z")

</div>

thanks advice from @yufeldman, dremio-env is updated :  
DREMIO\_JAVA\_EXTRA\_OPTS="-Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true -Djavax.net.debug=all"

and this time i am using the hive service account to run dremio, and config to use hive kerberos ticket with the embedded zk

dremio.conf  
services: {  
coordinator.embedded\_master\_zk.enabled: true,  
coordinator.embedded\_master\_zk.port: 12181  
}

services.kerberos.principal: “hive/en1-dev1-tbdp.dev.com@DEV.COM”  
services.kerberos.keytab.file.path: “/etc/security/keytabs/hive.service.keytab”

but the error is the same, and no much more info in the server.log to me  
[dremio-log.zip](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/6b081d66b75f10438ca5f564e642491a5e1f1fc6.zip) (13.4 KB)

Would someone have any ideas? thanks a ton.

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [December 7, 2017, 3:50pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/32 "2017-12-07T15:50:58Z")

</div>

I don’t think it is using kerberos debug. Can you check dremio execution command if those options

> [@forestlzj](#):
>
> -Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true -Djavax.net.debug=all

are included and included correctly - without quotes

```
ps -aef | grep Dremio

```

Also - do you have user kerberos principal (no host)? I would recommend using it in both dremio.conf (which is BTW not really needed - as long as you include core-site.xml with those specified) and while creating Hive source.

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 8, 2017, 2:47am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/33 "2017-12-08T02:47:24Z")

</div>

**without quotes:**

/opt/dremio-community-1.3/conf/dremio-env: line 78: -Dsun.security.spnego.debug=true: command not found  
starting dremio, logging to /opt/dremio-community-1.3/log/server.out

dremio@en1-dev1-tbdp:/opt/dremio-community-1.3$ ps -aef | grep Dremio  
dremio 75218 75146 70 10:22 pts/1 00:00:13 /opt/jdk1.8.0\_144/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/opt/dremio-community-1.3/log/server.gc -Ddremio.log.path=/opt/dremio-community-1.3/log -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/dremio-community-1.3/log -cp /opt/dremio-community-1.3/conf:/opt/dremio-community-1.3/jars/_:/opt/dremio-community-1.3/jars/ext/_:/opt/dremio-community-1.3/jars/3rdparty/\* com.dremio.dac.daemon.DremioDaemon dremio start

**with quotes**  
dremio 77744 77686 14 10:27 pts/1 00:00:11 /opt/jdk1.8.0\_144/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/opt/dremio-community-1.3/log/server.gc -Ddremio.log.path=/opt/dremi-community-1.3/log -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/dremio-community-1.3/log - **Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true -Djavax.net.debug=all** -cp /opt/dremio-community-1.3/conf:/opt/dremio-community-1.3/jars/_:/opt/dremio-community-1.3/jars/ext/_:/opt/dremio-community-1.3/jars/3rdparty/\* com.dremio.dac.daemon.DremioDaemon dremio start

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [December 8, 2017, 4:11pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/34 "2017-12-08T16:11:12Z")

</div>

ok. As much as I can tell from the log you posted following is the error:

> ```
> Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
> 
> ```
> 
> at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) ~[na:1.8.0\_131]  
> at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) ~[na:1.8.0\_131]  
> at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) ~[na:1.8.0\_131]  
> at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) ~[na:1.8.0\_131]  
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) ~[na:1.8.0\_131]  
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[na:1.8.0\_131]  
> at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ~[na:1.8.0\_131]  
> … 78 common frames omitted

But still there is no indication Kerberos debug is on.

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 9, 2017, 3:14am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/35 "2017-12-09T03:14:32Z")

</div>

wondering if kerberos config is right in the dremio.conf… as even using the hive service account , it has the same error

services.kerberos.principal: "hive/en1-dev1-tbdp.dev.com@DEV.COM"  
services.kerberos.keytab.file.path: “/etc/security/keytabs/hive.service.keytab”

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [December 10, 2017, 5:31pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/36 "2017-12-10T17:31:36Z")

</div>

Kerberos config in dremio.conf is not related to connection to Hive. It is for connection to HDFS or if you use HDFS for metadata.

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 12, 2017, 6:29am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/37 "2017-12-12T06:29:55Z")

</div>

it’s not able to connect to a kerberiosized hdfs as same error to me. Does the Community version support the connection to a securitied hadoop with kerberios?  
[Community Edition vs Enterprise Edition](https://www.dremio.com/enterprise-edition/)

 ![kerberos-dremio](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/ed27498163d37ba16fbaf78d3c432ebb0aabcc69.PNG)

---

<div class="post-metadata">

**Author:** ![forestlzj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/forestlzj/32/320_2.png) [@forestlzj](https://community.dremio.com/u/forestlzj)\
**Post date:** [December 27, 2017, 7:36am UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/38 "2017-12-27T07:36:34Z")

</div>

The Community version can connect to hive without kerberios enabled. but after kerberios enabled, it is failed.

---

<div class="post-metadata">

**Author:** ![oleksandr](https://avatars.discourse-cdn.com/v4/letter/o/dfb087/32.png) [@oleksandr](https://community.dremio.com/u/oleksandr)\
**Post date:** [April 16, 2019, 1:49pm UTC](https://community.dremio.com/t/connection-string-options-for-hive/167/39 "2019-04-16T13:49:17Z")

</div>

I’m trying to connect to Hive, normally I was using Beeline to connect to Hive (having username and password).  
Could you please advise what I need additionally to configure/check in order to connect to Hive in Dremio?

[Previous page](https://community.dremio.com/t/connection-string-options-for-hive/167.md?page=1)