Query via Hive2 with HDFS not on 8020 issue

Hi,

I’ve found several other topics related to this issue. As adviced in https://community.dremio.com/t/issue-while-accessing-hive I’ve added core-site.xml and hive-site.xml to:

  • <DREMIO_HOME>/conf

    dremio@dremio-1:/opt/dremio/conf$ ls -lah
    total 44K
    drwxr-xr-x 1 root root 4.0K Sep 15 08:36 .
    drwxr-xr-x 1 root root 4.0K Aug 27 08:05 …
    -rwxrwxrwx 1 root root 2.1K Sep 15 08:35 core-site.xml
    -rw-r–r-- 1 root root 2.5K Aug 27 07:30 dremio-env
    -rw-r–r-- 1 root root 914 Aug 27 07:30 dremio.conf
    -rwxrwxrwx 1 root root 3.1K Sep 15 08:35 hive-site.xml
    -rw-r–r-- 1 root root 1.8K Aug 27 07:30 logback-access.xml
    -rw-r–r-- 1 root root 2.2K Aug 27 07:30 logback-admin.xml
    -rw-r–r-- 1 root root 7.5K Aug 27 07:30 logback.xml

  • <DREMIO_HOME>/plugins/connectors/hive2.d

    dremio@dremio-1:/opt/dremio/plugins/connectors/hive2.d$ ls -lah
    total 16K
    drwxr-xr-x 2 root root 4.0K Sep 15 08:36 .
    drwxr-xr-x 1 root root 4.0K Sep 15 08:36 …
    -rwxrwxrwx 1 root root 2.1K Sep 15 08:35 core-site.xml
    -rwxrwxrwx 1 root root 3.1K Sep 15 08:35 hive-site.xml

In my specific situation HDFS is listening on port 9000.

core-site.xml and hive-site.xml contains:

<property><name>fs.defaultFS</name><value>hdfs://namenode:9000</value></property>

Additionally i set

Dremio lists the databases and tables, but as soon as I try to query I get the following message:

Call From dremio-1/172.23.0.11 to namenode:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

As you can see it still uses port 8020. As an attachment I’ve included the job profile.aa111dbc-f559-4683-9837-c0c7b8c9fc1e.zip (12.4 KB)

Any idea or hint where the namenode:8020, especially 8020, is comming from?

As an addition:

It seems to work for data persisted with Avro. But still throws this issue for Parquet data.