Authentication failure on arrow flight sql odbc driver

Hi,
Following is my setup. I run Dremio community edition of a standalone Ubuntu VM.

VM: Ubuntu 22.04 LTS
Dremio Version: dremio-community-4.9.1-202010230218060541-2e764ed0.tar.gz
arrow flight sql odbc driver: arrow-flight-sql-odbc-driver_0.9.0.116-2_amd64.deb

Following are the excerpts of dremio.conf file

services: {
  coordinator.enabled: true,
  coordinator.master.enabled: true,
  executor.enabled: true,
  flight.enabled: true,
  flight.ssl.enabled: false,
}

contents of /etc/odbc.ini file

[ODBC Data Sources]
Arrow_Flight_SQL_ODBC_DSN=Arrow Flight SQL ODBC Driver

[Arrow_Flight_SQL_ODBC_DSN]
Description=ODBC Driver DSN for Arrow Flight SQL developed by Dremio
Driver=Arrow Flight SQL ODBC Driver
host=0.0.0.0
port=32010
user=admin
password=<removed>
useEncryption=false
TrustedCerts=/opt/arrow-flight-sql-odbc-driver/lib64/cacerts.pem
UseSystemTrustStore=true

Note: The password does not contain any special characters

contents of /etc/odbcinst.ini

[Arrow Flight SQL ODBC Driver]
Description=ODBC Driver for Arrow Flight SQL developed by Dremio
Driver=/opt/arrow-flight-sql-odbc-driver/lib64/libarrow-odbc.so.0.9.0.116
FileUsage=1
UsageCount=1

In my Jupyter Notebook, I am using the following code to establish a connection

import pyodbc
cnxn = pyodbc.connect("DSN=Arrow_Flight_SQL_ODBC_DSN", autocommit=True)
cursor=cnxn.cursor()

The error I get is the following

---------------------------------------------------------------------------
Error                                     Traceback (most recent call last)
Input In [7], in <cell line: 2>()
      1 import pyodbc
----> 2 cnxn = pyodbc.connect("DSN=Arrow_Flight_SQL_ODBC_DSN", autocommit=True)
      3 cursor=cnxn.cursor()

Error: ('HY000', '[HY000] [unixODBC][Apache Arrow][Flight SQL] (100) Flight returned internal error, with message: There was an error servicing your request. (100) (SQLDriverConnect)')
---------------------------------------------------------------------------

I see the following in the Dremio server.log

---------------------------------------------------------------------------
2022-09-06 10:32:10,343 [flight-server-default-executor-16] ERROR o.a.a.flight.auth.ServerAuthWrapper - Error during authentication
java.lang.IllegalStateException: Requesting more messages than client sent.
        at org.apache.arrow.flight.auth.ServerAuthWrapper$AuthObserver$1.next(ServerAuthWrapper.java:106)
        at org.apache.arrow.flight.auth.ServerAuthWrapper$AuthObserver$1.next(ServerAuthWrapper.java:94)
        at org.apache.arrow.flight.auth.BasicServerAuthHandler.authenticate(BasicServerAuthHandler.java:55)
        at org.apache.arrow.flight.auth.ServerAuthWrapper.lambda$wrapHandshake$0(ServerAuthWrapper.java:58)
        at io.grpc.Context$1.run(Context.java:605)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
---------------------------------------------------------------------------

Am I missing something?
Any help is appreciated.

Thanks,
Ramu

@RamuR Welcome to Dremio Community!

This looks to be old. Please try again with any Dremio version >22.0

thanks @lenoyjacob. I was using the older version of Dremio. I switched to Dremio version >22.0 and it worked.

@lenoyjacob

I found the same error in Dremio version 23.1

server.log
+++++++++++++++++++++++++++++++++++++++++++++++
2022-12-08 10:42:02,546 [flight-server-default-executor-6] ERROR o.a.a.flight.auth.ServerAuthWrapper - Error during authentication
java.lang.IllegalStateException: Requesting more messages than client sent.
at org.apache.arrow.flight.auth.ServerAuthWrapper$AuthObserver$1.next(ServerAuthWrapper.java:106)
at org.apache.arrow.flight.auth.ServerAuthWrapper$AuthObserver$1.next(ServerAuthWrapper.java:94)
at org.apache.arrow.flight.auth.BasicServerAuthHandler.authenticate(BasicServerAuthHandler.java:55)
at org.apache.arrow.flight.auth.ServerAuthWrapper.lambda$wrapHandshake$0(ServerAuthWrapper.java:58)
at io.grpc.Context$1.run(Context.java:566)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)

dremio.conf
+++++++++++++++++++++++++++++++++++++++++++++++
flight.enabled: true,
flight.port: 32010,
flight.ssl.enabled: true,
flight.ssl.auto-certificate.enabled: false,
flight.ssl.keyStoreType: JKS,
flight.ssl.keyStore: “/opt/dremio/conf/keystore.jks”,
flight.ssl.keyStorePassword: “XXXXXXXX”,
flight.ssl.keyPassword: “XXXXXXXX”,

Background
+++++++++++++++++++++++++++++++++++++++++++++++

  1. Dremio version 23.1
  2. 1 coordinator and 3 executors
  3. oauth + ldap mode is used
  4. Client cacert has the server cert

Testing Result
+++++++++++++++++++++++++++++++++++++++++++++++
image

image

image

Questions
+++++++++++++++++++++++++++++++++++++++++++++++

  1. Anyone can help to fix this problem?
  2. How do I to find the correct configuration for setup Arrow Flight connection in both server and client sides?

Ming

I get a similar sort of error trying to connect Superset 2.0 with Dremio 23 on AWS Community edition.

ERROR: (builtins.NoneType) None
[SQL: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNKNOWN: ipv4:10.x.x.x:32010: Ssl handshake failed: SSL_ERROR_SSL: error:0A00010B:SSL routines::wrong version number]
(Background on this error at: Error Messages — SQLAlchemy 1.4 Documentation)

I have gotten the ssl versions on both dremio coordinator and the superset docker to be 1.1.1g (although the amzn linux ssl version says 1.1.1g FIPS), have set my server .pem file in the cacerts.pem. All still to no avail. keep getting the above error.

I almost wonder if some sort of bug/issue has crept into the 23.1 dremio version with how it interacts with the arrow flight odbc driver 0.9.1.168 ?

This finally worked for me. Was able to finally connect Superset 2.0 with AWS Dremio 23 comm. edition.

dremio+flight://admin:XXXXXXXXXX@x.x.x.x:32010/dremio?UseEncryption=false&disableCertificateVerification=false

The issue is resolved at least for me.