Dremio

# Unable to connect to Dremio through demio\_client using Jupyter Notebook on MacOS **URL:** \ **Category:** Dremio University\ **Created:** [August 30, 2022, 1:37pm UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604 "2022-08-30T13:37:19Z")\ **Posts on this page:** 9\ **Page:** 1

**Author:** ![BandanaPandey](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@BandanaPandey](https://community.dremio.com/u/BandanaPandey)\ **Post date:** [August 30, 2022, 1:37pm UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/1 "2022-08-30T13:37:19Z")

I am unable to connect to dremio using dremio\_client library on Jupyter NoteBook Below are the commands which I tried on Jupyter: ```auto from dremio_client import init client = init('~/.config/dremio_client/') results = client.query('SELECT * FROM demo_table') results ``` And this is the output which I am receiving, I have attached the screenshot below ![Screenshot 2022-08-30 at 7.00.25 PM](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/1/1294b46b09acb8697103ce61e9a57d2059663e83.png) Also I tried to debug it and I found that this api **api/v3/catalog** is the issue, it is returning HTML content, So I tried to access the same api using **curl** like this: ```auto curl -X GET http://localhost:9047/api/v3/catalog -H "Content-Type: application/json" -H "Authorization: user_token" ``` and it response of the above api was this: ```auto Dremio

``` It would be very helpful, if someone can help me out here. Thanks in advance ---

**Author:** ![lenoyjacob](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/lenoyjacob/32/3474_2.png) [@lenoyjacob](https://community.dremio.com/u/lenoyjacob)\ **Post date:** [August 31, 2022, 1:42am UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/2 "2022-08-31T01:42:34Z")

Hey @BandanaPandey, welcome to Dremio Community. Letting you know that the (unofficial) dremio\_client library hasn’t been updated in a while (potentially even deprecated). Here are a few ways instead (minimal code) to connect to Dremio: **Via ODBC (v22 and above only):** ```auto # !wget https://download.dremio.com/arrow-flight-sql-odbc-driver/arrow-flight-sql-odbc-driver-LATEST.x86_64.rpm # !sudo yum localinstall -y arrow-flight-sql-odbc-driver-LATEST.x86_64.rpm import pyodbc import pandas host = 'your_host' port = 32010 uid = 'your_user' pwd = 'your_pass' #If you're using a MacOS, the driver is located here -> /Library/Dremio/ODBC/lib/libarrow-flight-sql-odbc.dylib; the below is for linux: driver = "/opt/arrow-flight-sql-odbc-driver/lib64/libarrow-odbc.so.0.9.1.168" #Set UseEncryption accordingly below: cnxn=pyodbc.connect("Driver={};ConnectionType=Direct;HOST={};PORT={};AuthenticationType=Plain;UID={};PWD={}".format(driver,host,port,uid,pwd),autocommit=True,UseEncryption=False) sql = 'select * from sys.options limit 3' data = pandas.read_sql(sql, cnxn) print(data) ``` **Via REST:** See more Dremio APIs here: [Dremio](https://docs.dremio.com/software/rest-api/) ```auto #!pip install requests import requests import json import time host = 'your_host' port = '9047' uid = 'your_user' pwd = 'your_pass' #Authenticate headers = {'content-type': 'application/json'} response = requests.post('http://'+host+':'+port+'/apiv2/login', headers=headers, data = json.dumps({"userName":uid,"password":pwd})) headers={'content-type':'application/json', 'authorization':'_dremio'+response.json()['token']} #Query sql = 'select * from sys.options limit 3' job_id = requests.post('http://'+host+':'+port+'/api/v3/sql', headers=headers, data = json.dumps({"sql":sql})) #Wait till results; there are better ways to do this. For example, see how the old dremio client did it here: https://github.com/rymurr/dremio_client/blob/10e9b22f2cade17d25671b615a04b1c13752593f/dremio_client/util/query.py#L47... time.sleep(5) results = requests.get('http://'+host+':'+port+'/api/v3/job/'+job_id.json()['id']+'/results?offset=0&limit=500', headers=headers) #Print results.json() ``` **Via Arrow Flight:** ```auto #!pip install pyarrow import pyarrow from pyarrow import flight import pandas host = 'your_host' port = '32010' uid = 'your_user' pwd = 'your_pass' #Connect client = flight.FlightClient('grpc+tcp://' + host + ':' + port) #Authenticate bearer_token = client.authenticate_basic_token(uid, pwd) options = flight.FlightCallOptions(headers=[bearer_token]) #Query sql= "SELECT * FROM sys.memory" info = client.get_flight_info(flight.FlightDescriptor.for_command(sql),options) reader = client.do_get(info.endpoints[0].ticket, options) #Print df=reader.read_all() print(df) ``` Let me know if you need the ODBC script for Dremio versions lower than v22. ---

**Author:** ![BandanaPandey](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@BandanaPandey](https://community.dremio.com/u/BandanaPandey)\ **Post date:** [August 31, 2022, 7:24am UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/3 "2022-08-31T07:24:36Z")

Hey @lenoyjacob Thanks for the reply. I tried the same **via Arrow flight** on Mac ```auto import pyarrow from pyarrow import flight import pandas host = 'localhost' port = '31010' uid = 'bandana' pwd = 'dummy_pwd' #Connect client = flight.FlightClient('grpc+tcp://' + host + ':' + port) #Authenticate bearer_token = client.authenticate_basic_token(uid, pwd) options = flight.FlightCallOptions(headers=[bearer_token]) #Query sql= "SELECT * FROM sys.memory" info = client.get_flight_info(flight.FlightDescriptor.for_command(sql),options) reader = client.do_get(info.endpoints[0].ticket, options) #Print df=reader.read_all() print(df) ``` But I got this error: ![Screenshot 2022-08-31 at 12.54.07 PM](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/8/831ef662094a87c730fe09f8d80b5c9b22cd282f.png) ---

**Author:** ![BandanaPandey](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@BandanaPandey](https://community.dremio.com/u/BandanaPandey)\ **Post date:** [August 31, 2022, 8:23am UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/4 "2022-08-31T08:23:02Z")

Also @lenoyjacob when I tried via **REST** Api ```auto #!pip install requests import requests import json import time host = 'your_host' port = '9047' uid = 'your_user' pwd = 'your_pass' #Authenticate headers = {'content-type': 'application/json'} response = requests.post('http://'+host+':'+port+'/apiv2/login', headers=headers, data = json.dumps({"userName":uid,"password":pwd})) headers={'content-type':'application/json', 'authorization':'_dremio'+response.json()['token']} #Query sql = 'select * from sys.options limit 3' job_id = requests.post('http://'+host+':'+port+'/api/v3/sql', headers=headers, data = json.dumps({"sql":sql})) print(job_id.text) #Wait till results; there are better ways to do this. For example, see how the old dremio client did it here: https://github.com/rymurr/dremio_client/blob/10e9b22f2cade17d25671b615a04b1c13752593f/dremio_client/util/query.py#L47... time.sleep(5) results = requests.get('http://'+host+':'+port+'/api/v3/job/'+job_id.json()['id']+'/results?offset=0&limit=500', headers=headers) #Print results.json() ``` And **printed the job\_id** The print statement is returning this: ```auto Dremio

``` **Because of which it is throwing the below mentioned error:** ```auto --------------------------------------------------------------------------- JSONDecodeError Traceback (most recent call last) File /usr/local/lib/python3.10/site-packages/requests/models.py:971, in Response.json(self, **kwargs) 970 try: --> 971 return complexjson.loads(self.text, **kwargs) 972 except JSONDecodeError as e: 973 # Catch JSON-related errors and raise as requests.JSONDecodeError 974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError File /usr/local/lib/python3.10/site-packages/simplejson/ __init__.py:525, in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw) 521 if (cls is None and encoding is None and object_hook is None and 522 parse_int is None and parse_float is None and 523 parse_constant is None and object_pairs_hook is None 524 and not use_decimal and not kw): --> 525 return _default_decoder.decode(s) 526 if cls is None: File /usr/local/lib/python3.10/site-packages/simplejson/decoder.py:370, in JSONDecoder.decode(self, s, _w, _PY3) 369 s = str(s, self.encoding) --> 370 obj, end = self.raw_decode(s) 371 end = _w(s, end).end() File /usr/local/lib/python3.10/site-packages/simplejson/decoder.py:400, in JSONDecoder.raw_decode(self, s, idx, _w, _PY3) 399 idx += 3 --> 400 return self.scan_once(s, idx=_w(s, idx).end()) JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: JSONDecodeError Traceback (most recent call last) Input In [5], in () 18 #print(job_id.text) 19 20 # print(job_id.json()) 21 22 #Wait till results; there are better ways to do this. For example, see how the old dremio client did it here: https://github.com/rymurr/dremio_client/blob/10e9b22f2cade17d25671b615a04b1c13752593f/dremio_client/util/query.py#L47... 23 time.sleep(5) ---> 24 results = requests.get('http://'+host+':'+port+'/api/v3/job/'+job_id.json()['id']+'/results?offset=0&limit=500', headers=headers) File /usr/local/lib/python3.10/site-packages/requests/models.py:975, in Response.json(self, **kwargs) 971 return complexjson.loads(self.text, **kwargs) 972 except JSONDecodeError as e: 973 # Catch JSON-related errors and raise as requests.JSONDecodeError 974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError --> 975 raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) JSONDecodeError: Expecting value: line 1 column 1 (char 0) ``` ---

**Author:** ![lenoyjacob](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/lenoyjacob/32/3474_2.png) [@lenoyjacob](https://community.dremio.com/u/lenoyjacob)\ **Post date:** [August 31, 2022, 3:43pm UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/5 "2022-08-31T15:43:17Z")

@BandanaPandey What version of Dremio are you on? ---

**Author:** ![BandanaPandey](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@BandanaPandey](https://community.dremio.com/u/BandanaPandey)\ **Post date:** [August 31, 2022, 4:10pm UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/6 "2022-08-31T16:10:38Z")

Hey @lenoyjacob I have downloaded the community edition on mac, ![Screenshot 2022-08-31 at 9.38.42 PM](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/2/22ad3fca588dcb222f6de64d210a0fb2feff15cb.png) ---

**Author:** ![lenoyjacob](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/lenoyjacob/32/3474_2.png) [@lenoyjacob](https://community.dremio.com/u/lenoyjacob)\ **Post date:** [August 31, 2022, 4:29pm UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/7 "2022-08-31T16:29:14Z")

@BandanaPandey That’s possibly the culprit. v1.0.8 is old. Try again on something newer: v22.1.1 → [https://download.dremio.com/community-server/22.1.1-202208230402290397-a7010f28/dremio-community-22.1.1-202208230402290397-a7010f28.tar.gz](https://download.dremio.com/community-server/22.1.1-202208230402290397-a7010f28/dremio-community-22.1.1-202208230402290397-a7010f28.tar.gz) v21.2.0 → [https://download.dremio.com/community-server/21.2.0-202205262146080444-038d6d1b/dremio-community-21.2.0-202205262146080444-038d6d1b.tar.gz](https://download.dremio.com/community-server/21.2.0-202205262146080444-038d6d1b/dremio-community-21.2.0-202205262146080444-038d6d1b.tar.gz) ---

**Author:** ![BandanaPandey](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@BandanaPandey](https://community.dremio.com/u/BandanaPandey)\ **Post date:** [September 1, 2022, 6:10am UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/8 "2022-09-01T06:10:50Z")

Hey @lenoyjacob , Yes now the rest apis are working after I have installed the newer version. Thank You so much. Let me try with arrow flight. ---

**Author:** ![gakshat1107](https://avatars.discourse-cdn.com/v4/letter/g/da6949/32.png) [@gakshat1107](https://community.dremio.com/u/gakshat1107)\ **Post date:** [October 7, 2024, 12:36pm UTC](https://community.dremio.com/t/unable-to-connect-to-dremio-through-demio-client-using-jupyter-notebook-on-macos/9604/9 "2024-10-07T12:36:14Z")

hi @BandanaPandey , Were you able to connect through Arrow flight? Cause I’m also facing the same issue