Not able to connect Hive Source

1

I am trying to create the Hive Source from Dremio UI but I am not able to do so and getting errors in the logs. Can anyone help what might be wrong?

For now, I have added the below configs in the core-site.xml of Hadoop (Just for testing purposes) and restarted all services.

<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> fs.default.name hdfs://localhost:8301 hadoop.proxyuser.dremio.hosts * hadoop.proxyuser.dremio.groups * hadoop.proxyuser.dremio.users *

Error Trace:
2021-03-17 15:58:53,719 [start-Test] WARN c.d.h.t.transport.TIOStreamTransport - Error closing output stream.
java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at com.dremio.hive.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
at com.dremio.hive.thrift.transport.TSocket.close(TSocket.java:235)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:558)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.reconnect(HiveMetaStoreClient.java:373)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:146)
at com.sun.proxy.$Proxy202.getAllFunctions(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2265)
at com.sun.proxy.$Proxy202.getAllFunctions(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3646)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:238)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:223)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:369)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:312)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:292)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:268)
at com.dremio.exec.store.hive.HiveClientImpl.lambda$connect$0(HiveClientImpl.java:160)
at com.dremio.exec.store.hive.HiveClientImpl.lambda$doAsCommand$9(HiveClientImpl.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
at com.dremio.exec.store.hive.HiveClientImpl.doAsCommand(HiveClientImpl.java:367)
at com.dremio.exec.store.hive.HiveClientImpl.connect(HiveClientImpl.java:157)
at com.dremio.exec.store.hive.HiveClientImpl.createConnectedClient(HiveClientImpl.java:143)
at com.dremio.exec.store.hive.HiveStoragePlugin.createConnectedClient(HiveStoragePlugin.java:998)
at com.dremio.exec.store.hive.HiveStoragePlugin.start(HiveStoragePlugin.java:853)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$1(ManagedStoragePlugin.java:523)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:591)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-03-17 15:58:56,191 [start-Test] WARN c.d.e.catalog.ManagedStoragePlugin - Error starting new source: Test
java.lang.reflect.UndeclaredThrowableException: null
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1862)
at com.dremio.exec.store.hive.HiveClientImpl.doAsCommand(HiveClientImpl.java:367)
at com.dremio.exec.store.hive.HiveClientImpl.connect(HiveClientImpl.java:157)
at com.dremio.exec.store.hive.HiveClientImpl.createConnectedClient(HiveClientImpl.java:143)
at com.dremio.exec.store.hive.HiveStoragePlugin.createConnectedClient(HiveStoragePlugin.java:998)
at com.dremio.exec.store.hive.HiveStoragePlugin.start(HiveStoragePlugin.java:853)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:591)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: com.dremio.hive.thrift.transport.TTransportException
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:228)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:369)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:312)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:292)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:268)
at com.dremio.exec.store.hive.HiveClientImpl.lambda$connect$0(HiveClientImpl.java:160)
at com.dremio.exec.store.hive.HiveClientImpl.lambda$doAsCommand$9(HiveClientImpl.java:369)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
… 11 common frames omitted
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: com.dremio.hive.thrift.transport.TTransportException
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3649)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:238)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:223)
… 20 common frames omitted
Caused by: com.dremio.hive.thrift.transport.TTransportException: null
at com.dremio.hive.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at com.dremio.hive.thrift.transport.TTransport.readAll(TTransport.java:86)
at com.dremio.hive.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at com.dremio.hive.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at com.dremio.hive.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at com.dremio.hive.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_all_functions(ThriftHiveMetastore.java:3716)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_all_functions(ThriftHiveMetastore.java:3704)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllFunctions(HiveMetaStoreClient.java:2328)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
at com.sun.proxy.$Proxy202.getAllFunctions(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2265)
at com.sun.proxy.$Proxy202.getAllFunctions(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3646)
… 22 common frames omitted
2021-03-17 15:58:56,205 [start-Test] INFO c.d.s.s.LocalSchedulerService - Cancelling task metadata-refresh-wakeup-Test

2

@lyashninan

What version of Hive is this? What version of Dremio is this?

If it Dremio 4.2 or later, you have to either copy or symlink all the xml files under the conf folder to “/plugins/connectors/hive2.d/”

3

Hive version - 2.3.8
Dremio - 4.8.

4

@lyashninan

Have you done the above step?

5

Should it be /plugins/connectors/hive2.d/ or /plugins/connectors/hive2.d/conf to which we need to copy the XML files?

6

@lyashninan

Try these steps

cd <dremio-root>/plugins/connectors
ln -s <dremio-root>/conf <dremio-root>/plugins/connectors/hive2.d/conf
7

@lyashninan Sorry had a typo, edited my message above, please check again

8

This has to be done after copying the xml files or a symlink can be created without copying as well?

9

@lyashninan

It is either or, no need to do both

10

I have made the changes but still, I am getting the same error. Is there anything else which I might be missing on this?

11

@lyashninan

Let us try two things

  • Review metastore log at the time of this failure as we see a metastore exception
    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: com.dremio.hive.thrift.transport.TTransportException
    at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3649)
    at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:238)
    at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:223)
    … 20 common frames omitted
  • Let us try to connect to the metastore from Hive shell on the Dremio coordinator
  • Can you confirm if the JDK version on the Dremio coordinator is 8
  • From Hive shell, can you please make sure you are on Hive 2.x and not 1.x
12

Hive Version is 2.3.8
hive> select version();
OK
2.3.8 rf1e87137034e4ecbe39a859d4ef44319800016d7
Time taken: 8.172 seconds, Fetched: 1 row(s)

JAVA version is 1.8
java -version
java version “1.8.0_172”
Java™ SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot™ 64-Bit Server VM (build 25.172-b11, mixed mode)

But I quite didn’t follow on the part of connecting to the metastore from hive shell on the dremio coordinator. Can you please elaborate lil more to me?

13

@lyashninan

Logon to the Dremio coordinator and invoke Hive shell from there and run query a Hive table, if that works then try the same from Dremio and check metastore logs