Connecting Dremio to Apache-superset

Dremio University
1

Hello everyone, I have a problem when trying to connect Dremio version 4.1.3-202001022113020736-53142377 to apache-Superset version , i tried the given SQLAlchemy URI: “dremio+pyodbc://[Dremio username]:[Dremio password]@[Dremio host]:31010/dremio” and “dremio//[Dremio username]:[Dremio password]@[Dremio host]:31010/dremio” , from this site “https://www.dremio.com/tutorials/dremio-apache-superset/” but it showed error in connection. Help me to resolve this issue!!!

**ERROR: {“error”: “Connection failed!\n\nThe error message returned was:\nCan’t load plugin: sqlalchemy.dialects:dremio.pyodbc”, “stacktrace”: "Traceback (most recent call last):\n File “/home/rrscw/venv/lib/python3.6/site-packages/superset/views/core.py”, line 1702, in testconn\n engine = database.get_sqla_engine(user_name=username)\n File “/home/rrscw/venv/lib/python3.6/site-packages/superset/utils/core.py”, line 131, in call\n value = self.func(*args, **kwargs)\n File “/home/rrscw/venv/lib/python3.6/site-packages/superset/models/core.py”, line 911, in get_sqla_engine\n return create_engine(url, **params)\n File “/home/rrscw/venv/lib/python3.6/site-packages/sqlalchemy/engine/init.py”, line 479, in create_engine\n return strategy.create(*args, kwargs)\n File “/home/rrscw/venv/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py”, line 61, in create\n entrypoint = u._get_entrypoint()\n File “/home/rrscw/venv/lib/python3.6/site-packages/sqlalchemy/engine/url.py”, line 172, in _get_entrypoint\n cls = registry.load(name)\n File “/home/rrscw/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py”, line 240, in load\n “Can’t load plugin: %s:%s” % (self.group, name)\nsqlalchemy.exc.NoSuchModuleError: Can’t load plugin: sqlalchemy.dialects:dremio.pyodbc\n"}

Tried the following command in the virtual environment
pip install sqlalchemy_dremio, according to the instruction given in the Dremio connect to MetaBase or Apache Superset as visualization tools but facing the following error

sqlalchemyDremio
sqlalchemyDremio805×754 28 KB

2

Dremio is available in the master branch of superset: https://github.com/apache/incubator-superset. Assuming you have cloned that and installed all the dependencies. Based on your error you have not installed the following (check for the exact name based on your linux distro). These two are required for pyodbc:

unixODBC
unixODBC-devel

Connection string example: dremio://dremio:dremio123@localhost:31010/dremio

1 Like
3

Hi there, thanks for the reply.
I tried as you suggested, I installed unixODBC and unixODBC_dev in dremio and superset, running on separate VM but got the error as below:

ERROR: {“error”: “Connection failed!\n\nThe error message returned was:\n(pyodbc.Error) (‘01000’, “[01000] [unixODBC][Driver Manager]Can’t open lib ‘/opt/dremio-odbc/lib64/libdrillodbc_sb64.so’ : file not found (0) (SQLDriverConnect)”)\n(Background on this error at: http://sqlalche.me/e/dbapi)”, “stacktrace”: “Traceback (most recent call last):\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 2276, in _wrap_pool_connect\n return fn()\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 303, in unique_connection\n return _ConnectionFairy._checkout(self)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 760, in _checkout\n fairy = _ConnectionRecord.checkout(pool)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 492, in checkout\n rec = pool._do_get()\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/impl.py”, line 238, in _do_get\n return self._create_connection()\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 308, in _create_connection\n return _ConnectionRecord(self)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 437, in init\n self.__connect(first_connect_check=True)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 639, in __connect\n connection = pool._invoke_creator(self)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py”, line 114, in connect\n return dialect.connect(*cargs, **cparams)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy_dremio/base.py”, line 200, in connect\n return self.dbapi.connect(*cargs, **cparams)\npyodbc.Error: (‘01000’, “[01000] [unixODBC][Driver Manager]Can’t open lib ‘/opt/dremio-odbc/lib64/libdrillodbc_sb64.so’ : file not found (0) (SQLDriverConnect)”)\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File “/home/jecrc/venv/lib/python3.6/site-packages/superset/views/core.py”, line 1705, in testconn\n with closing(engine.connect()) as conn:\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 2209, in connect\n return self._connection_cls(self, **kwargs)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 103, in init\n else engine.raw_connection()\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 2307, in raw_connection\n self.pool.unique_connection, _connection\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 2280, in _wrap_pool_connect\n e, dialect, self\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 1547, in _handle_dbapi_exception_noconnection\n util.raise_from_cause(sqlalchemy_exception, exc_info)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py”, line 398, in raise_from_cause\n reraise(type(exception), exception, tb=exc_tb, cause=cause)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py”, line 152, in reraise\n raise value.with_traceback(tb)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 2276, in _wrap_pool_connect\n return fn()\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 303, in unique_connection\n return _ConnectionFairy._checkout(self)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 760, in _checkout\n fairy = _ConnectionRecord.checkout(pool)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 492, in checkout\n rec = pool._do_get()\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/impl.py”, line 238, in _do_get\n return self._create_connection()\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 308, in _create_connection\n return _ConnectionRecord(self)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 437, in init\n self.__connect(first_connect_check=True)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py”, line 639, in __connect\n connection = pool._invoke_creator(self)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py”, line 114, in connect\n return dialect.connect(*cargs, **cparams)\n File “/home/jecrc/venv/lib/python3.6/site-packages/sqlalchemy_dremio/base.py”, line 200, in connect\n return self.dbapi.connect(*cargs, **cparams)\nsqlalchemy.exc.DBAPIError: (pyodbc.Error) (‘01000’, “[01000] [unixODBC][Driver Manager]Can’t open lib ‘/opt/dremio-odbc/lib64/libdrillodbc_sb64.so’ : file not found (0) (SQLDriverConnect)”)\n(Background on this error at: http://sqlalche.me/e/dbapi)\n”}

4

Have you installed Dremio ODBC driver? https://download.dremio.com/odbc-driver/1.4.2.1003/dremio-odbc-1.4.2.1003-1.x86_64.rpm

Please verify if the above file exists.

We put together a script and a blog post if you want to try it on CentOS: Visualize your data lake with Apache Superset and Dremio | Dremio