User: xxxxx is not allowed to impersonate hdfs

1

Hi
I just config core-site
hadoop.proxyuser.xxxx.hosts=*
hadoop.proxyuser.xxxx.groups=*
hadoop.proxyuser.xxxx.users=*

I got User: xxxxx is not allowed to impersonate hdfs

Could you help me?
Regards

2

Make sure that you deploy the new configuration on both the client (Dremio server) and the HDFS server. You also need to run a hdfs admin command to refresh impersonation configuration for HDFS.

3

Hi
Could you tell me how to perform these steps?

Regards

4

Because there are multiple Hadoop distribution, I cannot give you precise instructions. You should check with your Hadoop vendor, or if you are using the Apache Hadoop distribution, details are available at https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Superusers.html and https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#dfsadmin

5

My vendor is hortonworks.
Could you help my with:

  • New configuration on both the client (Dremio server) and the HDFS server.
  • You also need to run a hdfs admin command to refresh impersonation configuration for HDFS.
6

If you made impersonation work for everything else on your YARN cluster but Dremio try following:

In dremio-env file add following line:

export DREMIO_CLASSPATH_USER_FIRST=<your_distro_path_to_tarn_configuration>/etc/hadoop

Essentially set it to the directory where all Haddop *-site.xml files are located.

7

Hi
dremio-env
export DREMIO_CLASSPATH_USER_FIRST=/etc/hadoop/conf.backup/core-site.xml
export DREMIO_CLASSPATH_USER_FIRST=/etc/hadoop/2.6.2.0-205/0/core-site.xml

But did not work
Could you help me?

8

It should be a directory with those files, not files explicitly, or if you want it to be explicit you need to chain it:

export DREMIO_CLASSPATH_USER_FIRST=/etc/hadoop/conf.backup/core-site.xml
export DREMIO_CLASSPATH_USER_FIRST=$DREMIO_CLASSPATH_USER_FIRST:/etc/hadoop/2.6.2.0-205/0/core-site.xml

But I strongly recommend directory versus set of all arbitrary core-site.xml files you have

9

I chain to:
export DREMIO_CLASSPATH_USER_FIRST=/etc/hadoop/conf.backup/core-site.xml
export DREMIO_CLASSPATH_USER_FIRST=$DREMIO_CLASSPATH_USER_FIRST:/etc/hadoop/2.6.2.0-205/0/core-site.xml

But did not work
User: Dario is not allowed to impersonate hdfs

10
  1. does any of those core-site.xml files have all the necessary setup according to:
  1. Did you restart Dremio after that?
  2. Do you run Dremio as hdfs or your user and login to Dremio as hdfs?
11

1-
hadoop.proxyuser.dario.groups
*

<property>
  <name>hadoop.proxyuser.dario.hosts</name>
  <value>*</value>
</property>

<property>
  <name>hadoop.proxyuser.dario.users</name>
  <value>*</value>
</property>

2- I restarted Dremio
3- I’d like to use Dremio with hive.

12
  1. Please make sure you have any other daemon like applications that require impersonation running fine - like oozie, HiveServer2, etc.
    If it is not the case you need to make sure impersonation is working on your hadoop cluster BEFORE dealing with Dremio
  2. If/when #1 is resolved try to start Dremio with user “dario” and use it as the first user you create in Dremio.
  3. If/when #2 works you can create more users in Dremio to make sure impersonation works for them
13

I could fix my problem and it works, Thanks