ODBC with Zookeeper Quorum

1

Hi,
I configured 2-node Dremio coordinator with external ZooKeeper, and I can use JDBC with ZooKeeper Quorum to connect Dremio, but I got connection fail with ODBC ZooKeeper Quorum.

Here are my settings

dremio.conf ( I have multiple Dremio clusters in same ZooKeeper so I configured the path)
zookeeper: “host1.aa.com:2181/ho/ha/dremio,host2.aa.com:2181/ho/ha/dremio,host3.aa.com:2181/ho/ha/dremio,host4.aa.com:2181/ho/ha/dremio,host5.aa.com:2181/ho/ha/dremio”

JDBC connection (connect success !!! )
jdbc:dremio:zk=host1.aa.com:2181/ho/ha/dremio,host2.aa.com:2181/ho/ha/dremio,host3.aa.com:2181/ho/ha/dremio,host4.aa.com:2181/ho/ha/dremio,host5.aa.com:2181/ho/ha/dremio

ODBC connection (throw error)
Quorm: host1.aa.com:2181,host2.aa.com:2181,host3.aa.com:2181,host4.aa.com:2181,host5.aa.com:2181
Cluster ID:/ho/ha/dremio/coordinator

I wonder if i use correct cluster id for ODBC. I already tested below value, but still got error
/ho/ha/dremio/
/ho/ha/
/ho/

The ODBC driver is the latest windows 64-bit driver

Thanks,
James

2

@jamescheng,

What error does the ODBC connection throw with the quorum? Can you supply the error stack?

3

Hi @ben
The error message is

FAILED!

[Dremio][Connector] (10) Failure occurred while trying to connect to zk:2181=host1.aa.com,host2.aa.com:2181,host3.aa.com:2181,host4.aa.com:2181,host5.aa.com:2181/ho/ha/dremio/coordinator

Thanks,
James

4

@jamescheng,

Based on the error, it seems that the ODBC driver is maybe parsing the connection string incorrectly. Assuming that you have a ZKnode for one of the cluster coordinators at /ho/ha/dremio/coordinator can you try pointing to just one member of the quorum? If you are using the windows ODBC configurator that would look like:

Quorum: host1.aa.com:2181
Cluster ID: /ho/ha/dremio/coordinator
5

@ben,

I tried to use just one member of quorum, but it still threw error.
FAILED!

[Dremio][Connector] (10) Failure occurred while trying to connect to zk:2181=host1.aa.com/ho/ha/dremio/coordinator

The only one can be success is that if I use /dremio/coordinator as cluster id. However, the /dremio/coordinator is my another Dremio cluster. ( I have multiple Dremio cluster that using same zookeeper)

Thanks,
James

6

@jamescheng, do you have access to ZK shell?
Can you confirm the path in Zookeeper to the other Dremio coordinator in the other cluster?

7

@ben
Here is the path of my one Dremio Cluster (only 1 coordinator). This is working for JDBC and ODBC that using ZooKeeper Quorum
[zk: host1.aa.com:2181(CONNECTED) 9] ls /dremio
[semaphore, leader-latch, coordinator, executor, master]
[zk: host1.aa.com:2181(CONNECTED) 10] ls /dremio/coordinator
[cc86c318-c6dd-4547-8b79-d4473154dc76]

Here is the path of Dremio Cluster (2 coordinators). This is working for JDBC(with ZK Quorum), but not working with ODBC (with ZK Quorum)
[zk: host1.aa.com:2181(CONNECTED) 6] ls /ho
[ha]
[zk: host1.aa.com:2181(CONNECTED) 7] ls /ho/ha
[dremio,host2.aa.com:2181]
[zk: host1.aa.com:2181(CONNECTED) 8] ls /ho/ha/dremio,host2.aa.com:2181
[ho]

After the comparison, I can see this is different. I wonder if it is regarding the ZooKeeper setting in dremio.conf. I put the path at each node in dremio.conf, should I put the path only at the last ZK node ?

my current setting
“host1.aa.com:2181/ho/ha/dremio,host2.aa.com:2181/ho/ha/dremio,host3.aa.com:2181/ho/ha/dremio,host4.aa.com:2181/ho/ha/dremio,host5.aa.com:2181/ho/ha/dremio”

should i configure it ? I remember it can’t work

“host1.aa.com:2181,host2.aa.com:2181,host3.aa.com:2181,host4.aa.com:2181,host5.aa.com:2181/ho/ha/dremio”