Does Dremio Cloud support Arrow Flight?

Dremio Cloud
1

Hi! I’ve been able to connect to my Dremio Cloud instance with JDBC. Unfortunately, I’m having trouble connecting via Arrow Flight. Is this option enabled for Dremio Cloud?

My python script is as follows and I am using dremio_client:

from dremio_client import init
import pandas as pd

query = 'select 1'
client = init(simple_client=True)
results = client.query(self, query)
pd.DataFrame(results)

I then run:

DREMIO_HOSTNAME=data.dremio.cloud DREMIO_PORT=32010 DREMIO_AUTH_USERNAME='$token' DREMIO_AUTH_PASSWORD=<PersonalAccessToken> python3 -m flight_example

and get an error like:

requests.exceptions.ConnectTimeout: HTTPConnectionPool(host=‘data.dremio.cloud’, port=32010): Max retries exceeded with url: /apiv2/login (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb1f51d9150>, ‘Connection to data.dremio.cloud timed out. (connect timeout=10)’))

I’ve also tried to use the example here, e.g.

python3 example.py -query 'SELECT 1' -host data.dremio.cloud -port 32010 -authToken  <PersonalAccessToken>

and I then get the error:

OSError: gRPC returned unavailable error, with message: failed to connect to all addresses. Detail: Unavailable

which appears to be caused when the server is not accepting connections from Apache Flight.

Is the issue just that Dremio Cloud doesn’t support Arrow Flight?

2

Hi redgeoff,

Dremio certainly supports Arrow Flight. Here is our guide for getting started: Dremio

As to your issues, we will investigate and get back to you.

1 Like
3

Appreciate it. I would suspect that the “Dremio Software” edition has Arrow Flight support, but does “Dremio Cloud” have support for Arrow Flight?

4

Hi @redgeoff ,
Per the link @James_Gupta sent out, you should be using port 443 and enable encryption to use Arrow Flight with Dremio Cloud.

1 Like
5

Thank you @jduong!

I was able to get the arrow-flight-client-examples to work by changing the port, e.g.

python3 example.py -query 'SELECT 1' -host data.dremio.cloud -port 443 -tls -authToken <PersonalAccessToken>

Unfortunately, this didn’t seem to work for my code that uses dremio_client (see above). I tried:

DREMIO_HOSTNAME=data.dremio.cloud DREMIO_PORT=443 DREMIO_AUTH_USERNAME='$token' DREMIO_AUTH_PASSWORD=<PersonalAccessToken> DREMIO_SSL=true python3 -m flight_example

and received the error:

requests.exceptions.HTTPError: 415 Client Error: Unsupported Media Type for url: https://data.dremio.cloud:443/apiv2/login

I’m guessing this might just be a problem with dremio_client. I’ll post here if I find a workaround. (At least I know the arrow-flight-client-examples code works)