Fight connection is failing after upgrading to 17

1

FlightUnauthenticatedError: gRPC returned unauthenticated error, with message: Unauthenticated (invalid or missing auth token)
The same worked in 15.8. Did anyone else run into this issue? Hopefully there is a fix.

2

@Shirisha Even though this works in 15.x, it seems like a token issue, have we validated we do not have an expired token?

3

Yes token is not expired. The reason i am confidently saying this is because we are generating the token in the code every time call is made. Once reverted back, it worked with same code base.

4

Hi @Shirisha ,

Can you provide an example of the code you’re using to connect to Flight?

5

Sure here you go:
Pre reqs:
pip install pyarrow==0.15.0
pip install dremio-client
pip freeze | grep pyarrow
pip freeze | grep dremio-client

import sys, os
from dremio_client import init
from dremio_client import flight
import time
import pandas as pd

client = flight.connect(hostname=‘YOURHOST’, port=PORTNUMBER, username=‘YOURUSERNAME’, password=‘YOUR PWD’, tls_root_certs_filename=None)
sql= ‘SELECT * FROM sys.memory’
%%time
df = flight.query(sql, client=client)
df.head(10)

6

Flight’s auth logic doesn’t look to have changed between 15 and 17. Did you upgrade your cluster from 15 to 17 or create a new installation? If you created a new installation, did you login through the web UI to create the initial user?

7

web UI worked and all our VDSs worked as well. Its just that flight didnt.

8

Does anyone else ran into this issue?

9

its a cluster upgrade. dremio-services-arrow-flight-15.0.0-202103312106020527-0be9c719.jar and dremio-flight-connector-0.23.0-SNAPSHOT.jar are the jar files i see for Arrow flight. Do you think i need to get new ones?

10

Does anyone else ran into this issue? It is happening in both EE and CE dremio. Is this a bug?

11

This may be a dremio-client issue (possibly this one: Updates needed for newest Dremio version ¡ Issue #224 ¡ rymurr/dremio_client ¡ GitHub)

Try and see if following code works? I’m using pyarrow v5.0.0 and not using dremio-client.

pip install pyarrow==5.0.0

import pyarrow
from pyarrow import flight
import pandas

username = "yourusername"
password = "yourpassword"

client = flight.FlightClient('grpc+tcp://yourhost:32010')
bearer_token = client.authenticate_basic_token(username, password) 
options = flight.FlightCallOptions(headers=[bearer_token])

sql= "SELECT * FROM sys.memory"
info = client.get_flight_info(flight.FlightDescriptor.for_command(sql),options)
reader = client.do_get(info.endpoints[0].ticket, options)

df=reader.read_pandas()
df.head(10)

Code based from: arrow-flight-client-examples/example.py at main ¡ dremio-hub/arrow-flight-client-examples ¡ GitHub

12

Seems like the auth mode changed, from legacy.arrow.flight.auth to arrow.flight.auth2

Which has been addressed by me in this pull:

1 Like