How connect from Dremio to Hive Hadoop (without Ranger)

#1

How can I connect from Dremio to Hive Hadoop (without Ranger)?

#2

Hi @oleksandr

To connect hive,
you can select Storage Based with User Impersonation in the Authorization And provide the Hive Metastore Host hostname.

In the advanced options set this property

name: hive.server2.enable.doAs
value: false

#3

Hi Venugopal,
Thanks for your reply.
I’m looking for possibility to login using hive username and password. Is there any way to connect?

#4

Hi @oleksandr

Dremio does not connect to hiveserver2 , we connect to metastore just to get the schema information and directly access the file on HDFS.

#5

Hi Venugopal,

If I’m connecting

I can see tables but once I’m accessing data I’m getting the error:

Failure while retrieving dataset [Hive_test.acquire_v.ad_user_roles].

Please advise.

Thanks,

Oleksandr

#6

Hi @oleksandr

Can you check at the Profile for the ERROR details, if not please check at the server.log when the query runs.

#7

What is the file name I should look inside?

#8

@oleksandr It is Server.log.

You can check the error stack in the job profile as well under “error” field

@venu

#9

I have found the following:

2019-05-08 14:57:27,698 [metadata-refresh-Hive] WARN c.dremio.exec.store.hive.HiveClient - Failure to run Hive command. Will retry once.

org.apache.hadoop.hive.metastore.api.MetaException: org.apache.hadoop.security.AccessControlException: Permission denied: user=username, access=EXECUTE, inode="/apps/hive/warehouse/mdss_datamart_work.db/awb_mdr_metrics_z2":hdpmdss:hdpmdss:drwxr-x—

what is the interesting that it’s not the table I’m accessing

@oleksandr

#10

@oleksandr

can you attach the server.log and profile,will check and let you know.

#11

There was no job executed

(Attachment server.log is missing)

#12

Resent

server.zip (194 KB)

#13

Can you tell me the table you are tryng to access.

from the log i can see access denied for the user “altohnat”. I believe the user is notpart of the group “hdpmass”.

2019-05-08 17:03:36,985 [metadata-refresh-Hive] INFO c.dremio.exec.store.hive.HiveClient - Failure while trying to read table pre_otp_load_z2_all_file from db mdss_datamart_load
org.apache.hadoop.hive.metastore.api.MetaException: org.apache.hadoop.security.AccessControlException: Permission denied: user=altohnat, access=EXECUTE, inode="/data/mdss/datamart/load/
otp/z2/preload_all_file":hdpmdss:hdpmdss:drwx------
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:292)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:238)

I am able to repro a similar issue by altering the file permissions under the warehouse directory.

{code}
(org.apache.hadoop.security.AccessControlException) Permission denied: user=dremio, access=READ, inode="/apps/hive/warehouse/voter_text/voterhive.tsv":hive:hadoop:-rwx------
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:252)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
{code}

Permissions on the HDFS:

[hive@venumaprclinet dvenu]$ hadoop fs -ls /apps/hive/warehouse/voter_text/voterhive.tsv
-rwx------ 3 hive hadoop 14621 2019-05-02 07:15 /apps/hive/warehouse/voter_text/voterhive.tsv

Can you try by changing the permissions on the HDFS or add the user to the group and try.

Thanks
@Venugopal_Menda

#14

Table I’m accessing:

I will revert to you as soon as I have changed permissions for my user…. It takes time.

Thanks a lot