Fight connection is failing after upgrading to 17

FlightUnauthenticatedError: gRPC returned unauthenticated error, with message: Unauthenticated (invalid or missing auth token)
The same worked in 15.8. Did anyone else run into this issue? Hopefully there is a fix.

@Shirisha Even though this works in 15.x, it seems like a token issue, have we validated we do not have an expired token?

Yes token is not expired. The reason i am confidently saying this is because we are generating the token in the code every time call is made. Once reverted back, it worked with same code base.

Hi @Shirisha ,

Can you provide an example of the code you’re using to connect to Flight?

Sure here you go:
Pre reqs:
pip install pyarrow==0.15.0
pip install dremio-client
pip freeze | grep pyarrow
pip freeze | grep dremio-client

import sys, os
from dremio_client import init
from dremio_client import flight
import time
import pandas as pd

client = flight.connect(hostname=‘YOURHOST’, port=PORTNUMBER, username=‘YOURUSERNAME’, password=‘YOUR PWD’, tls_root_certs_filename=None)
sql= ‘SELECT * FROM sys.memory’
df = flight.query(sql, client=client)

Flight’s auth logic doesn’t look to have changed between 15 and 17. Did you upgrade your cluster from 15 to 17 or create a new installation? If you created a new installation, did you login through the web UI to create the initial user?

web UI worked and all our VDSs worked as well. Its just that flight didnt.

Does anyone else ran into this issue?

its a cluster upgrade. dremio-services-arrow-flight-15.0.0-202103312106020527-0be9c719.jar and dremio-flight-connector-0.23.0-SNAPSHOT.jar are the jar files i see for Arrow flight. Do you think i need to get new ones?

Does anyone else ran into this issue? It is happening in both EE and CE dremio. Is this a bug?

This may be a dremio-client issue (possibly this one: Updates needed for newest Dremio version · Issue #224 · rymurr/dremio_client · GitHub)

Try and see if following code works? I’m using pyarrow v5.0.0 and not using dremio-client.

pip install pyarrow==5.0.0
import pyarrow
from pyarrow import flight
import pandas

username = "yourusername"
password = "yourpassword"

client = flight.FlightClient('grpc+tcp://yourhost:32010')
bearer_token = client.authenticate_basic_token(username, password) 
options = flight.FlightCallOptions(headers=[bearer_token])

sql= "SELECT * FROM sys.memory"
info = client.get_flight_info(flight.FlightDescriptor.for_command(sql),options)
reader = client.do_get(info.endpoints[0].ticket, options)


Code based from: arrow-flight-client-examples/ at main · dremio-hub/arrow-flight-client-examples · GitHub

Seems like the auth mode changed, from legacy.arrow.flight.auth to arrow.flight.auth2

Which has been addressed by me in this pull:

1 Like