How can I connect from Dremio to Hive Hadoop (without Ranger)?
Hi @oleksandr
To connect hive,
you can select Storage Based with User Impersonation in the Authorization And provide the Hive Metastore Host hostname.
In the advanced options set this property
name: hive.server2.enable.doAs
value: false
Hi Venugopal,
Thanks for your reply.
I’m looking for possibility to login using hive username and password. Is there any way to connect?
Hi @oleksandr
Dremio does not connect to hiveserver2 , we connect to metastore just to get the schema information and directly access the file on HDFS.
Hi Venugopal,
If I’m connecting
I can see tables but once I’m accessing data I’m getting the error:
Failure while retrieving dataset [Hive_test.acquire_v.ad_user_roles].
Please advise.
Thanks,
Oleksandr
Hi @oleksandr
Can you check at the Profile for the ERROR details, if not please check at the server.log when the query runs.
@oleksandr It is Server.log.
You can check the error stack in the job profile as well under “error” field
@venu
I have found the following:
2019-05-08 14:57:27,698 [metadata-refresh-Hive] WARN c.dremio.exec.store.hive.HiveClient - Failure to run Hive command. Will retry once.
org.apache.hadoop.hive.metastore.api.MetaException: org.apache.hadoop.security.AccessControlException: Permission denied: user=username, access=EXECUTE, inode="/apps/hive/warehouse/mdss_datamart_work.db/awb_mdr_metrics_z2":hdpmdss:hdpmdss:drwxr-x—
what is the interesting that it’s not the table I’m accessing
can you attach the server.log and profile,will check and let you know.
Resent
server.zip (194 KB)
Can you tell me the table you are tryng to access.
from the log i can see access denied for the user “altohnat”. I believe the user is notpart of the group “hdpmass”.
2019-05-08 17:03:36,985 [metadata-refresh-Hive] INFO c.dremio.exec.store.hive.HiveClient - Failure while trying to read table pre_otp_load_z2_all_file from db mdss_datamart_load
org.apache.hadoop.hive.metastore.api.MetaException: org.apache.hadoop.security.AccessControlException: Permission denied: user=altohnat, access=EXECUTE, inode="/data/mdss/datamart/load/
otp/z2/preload_all_file":hdpmdss:hdpmdss:drwx------
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:292)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:238)
I am able to repro a similar issue by altering the file permissions under the warehouse directory.
{code}
(org.apache.hadoop.security.AccessControlException) Permission denied: user=dremio, access=READ, inode="/apps/hive/warehouse/voter_text/voterhive.tsv":hive:hadoop:-rwx------
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:252)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
{code}
Permissions on the HDFS:
[hive@venumaprclinet dvenu]$ hadoop fs -ls /apps/hive/warehouse/voter_text/voterhive.tsv
-rwx------ 3 hive hadoop 14621 2019-05-02 07:15 /apps/hive/warehouse/voter_text/voterhive.tsv
Can you try by changing the permissions on the HDFS or add the user to the group and try.
Thanks
@Venugopal_Menda
Table I’m accessing:
I will revert to you as soon as I have changed permissions for my user…. It takes time.
Thanks a lot
Hi Venugopal,
I’m running on Windows 10 Dremio server and Hive is running on Unix/Linux.
I have got one question user in the email chain below “altohnat” (from log files), is this user who started Dremio server or it’s user who logged in to Windows box? How can I change this user name?
Best regards,
Oleksandr
Hi @oleksandr
The ‘altohnat’ is the user who is accessing the table from Dremio UI on your Windows box.
Thanks
@Venugopal_Menda
It’s not really clear for me. As Dremio server is started using ‘altohnat’ user and I’m logged to Dremio UI using ‘john.smith’ user.
Thanks
Oleksandr