# Cannot create Dremio connection from Superset: Exception in RPC communication

**URL:** <https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636>\
**Category:** Uncategorized\
**Created:** [December 22, 2024, 11:00pm UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636 "2024-12-22T23:00:54Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![relio](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/relio/32/6217_2.png) [@relio](https://community.dremio.com/u/relio)\
**Post date:** [December 22, 2024, 11:00pm UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/1 "2024-12-22T23:00:54Z")

</div>

Hi all!  
I have a Dremio running docker instance that successfully access a Postgres db. I have also a running Superset docker instance created out of alexmerced/dremio-superset image that is not able to connect to Dremio due to this error:

```auto
20:43:25.168 [UserServer-1] ERROR com.dremio.exec.rpc.RpcExceptionHandler - Exception in RPC communication. Connection: /172.27.0.8:31010 <--> /172.27.0.7:39396 (user client). Closing connection.
io.netty.handler.codec.CorruptedFrameException: Expected to read a tag of 10 but actually received a value of 3. Happened after reading 0 message.

```

SQLALCHEMY URI is set as:

> dremio+flight://my-user:my-password.@172.27.0.8:31010/my-dremio-space?UseEncryption=false

Superset docker instance is able to see dremio docker instance as long as both instances are part of the same network. Superset docker instance is derived from alexmerced/dremio-superset so it has the SQL alchemy drivers.

dremio.conf looks like:

```auto
paths: {
  # the local path for dremio to store data.
  local: ${DREMIO_HOME}"/data"

  # the distributed path Dremio data including job results, downloads, uploads, etc
  #dist: "pdfs://"${paths.local}"/pdfs"
}

services: {
  coordinator.enabled: true,
  coordinator.master.enabled: true,
  executor.enabled: true,
  flight.use_session_service: true
}

```

I’ve added ‘coordinator.flight.enabled: true’ as I googled that it could be the reason for which I cannot establish a connection to Dremio from Superset.

Dremio docker instance has been created with:

> docker run -d --name=dremio --network=db-network -p 9047:9047 -p 31010:31010 -p 45678:45678 -p 32010:32010 -e DREMIO\_JAVA\_SERVER\_EXTRA\_OPTS=-Dpaths.dist=file:///opt/dremio/data/dist -v /my-path/conf:/opt/dremio/conf dremio/dremio-oss

Any suggestion? Other similar topics didn’t help me.

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 23, 2024, 5:24am UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/2 "2024-12-23T05:24:42Z")

</div>

@relio There seems to be a similar issue reported internally, I would like to verify if the stack is the same. Any chance you can send the profile?

---

<div class="post-metadata">

**Author:** ![relio](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/relio/32/6217_2.png) [@relio](https://community.dremio.com/u/relio)\
**Post date:** [December 23, 2024, 7:59am UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/3 "2024-12-23T07:59:32Z")

</div>

Mmh, which profile? 🙂

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [January 8, 2025, 5:35am UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/4 "2025-01-08T05:35:33Z")

</div>

@relio My bad, Are you able to send the logfile so we can see the full stack

---

<div class="post-metadata">

**Author:** ![relio](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/relio/32/6217_2.png) [@relio](https://community.dremio.com/u/relio)\
**Post date:** [January 8, 2025, 8:17am UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/5 "2025-01-08T08:17:30Z")

</div>

> [@relio](#):
>
> dremio+flight://my-user:my-password.@172.27.0.8:31010/my-dremio-space?UseEncryption=false

This is the relevant part of Superset log when I’m trying to connect to Dremio:

```auto
2025-01-08 08:12:36,121:DEBUG:superset.stats_logger:[stats_logger] (incr) test_connection_attempt
2025-01-08 08:12:36,129:DEBUG:superset.models.core:Database._get_sqla_engine(). Masked URL: dremio+flight://my-user:XXXXXXXXXX@172.27.0.8:31010/my-dremio-space?UseEncryption=false
2025-01-08 08:12:36,160:DEBUG:superset.stats_logger:[stats_logger] (incr) test_connection_error.DBAPIError
2025-01-08 08:12:36,163:DEBUG:superset.stats_logger:[stats_logger] (incr) DatabaseRestApi.test_connection.error
2025-01-08 08:12:36,163:WARNING:superset.views.base:SupersetErrorsException
Traceback (most recent call last):
  File "/app/superset/databases/commands/test_connection.py", line 163, in run
    raise DBAPIError(ex_str or None, None, None)
sqlalchemy.exc.DBAPIError: (builtins.NoneType) None
[SQL: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNAVAILABLE: ipv4:172.27.0.8:31010: Socket closed]
(Background on this error at: https://sqlalche.me/e/14/dbapi)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 127, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 121, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1518, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 255, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 93, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/databases/api.py", line 905, in test_connection
    TestConnectionDatabaseCommand(item).run()
  File "/app/superset/databases/commands/test_connection.py", line 192, in run
    raise SupersetErrorsException(errors) from ex
superset.exceptions.SupersetErrorsException: [SupersetError(message='(builtins.NoneType) None\n[SQL: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNAVAILABLE: ipv4:172.27.0.8:31010: Socket closed]\n(Background on this error at: https://sqlalche.me/e/14/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Dremio', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
172.27.0.1 - - [08/Jan/2025:08:12:36 +0000] "POST /api/v1/database/test_connection/ HTTP/1.1" 500 469 "http://localhost:8088/databaseview/list/?pageIndex=0&sortColumn=changed_on_delta_humanized&sortOrder=desc" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
2025-01-08 08:12:46,877:DEBUG:superset.stats_logger:[stats_logger] (incr) test_connection_attempt
2025-01-08 08:12:46,889:DEBUG:superset.models.core:Database._get_sqla_engine(). Masked URL: dremio+flight://my-user:XXXXXXXXXX@172.27.0.8:31010/my-dremio-space?UseEncryption=false
2025-01-08 08:12:46,892:DEBUG:superset.stats_logger:[stats_logger] (incr) test_connection_error.DBAPIError
2025-01-08 08:12:46,897:DEBUG:superset.stats_logger:[stats_logger] (incr) db_creation_failed.SupersetErrorsException
2025-01-08 08:12:46,903:DEBUG:superset.stats_logger:[stats_logger] (incr) DatabaseRestApi.post.error
2025-01-08 08:12:46,903:DEBUG:superset.stats_logger:[stats_logger] (timing) DatabaseRestApi.post.time | 31.010183000034885 
172.27.0.1 - - [08/Jan/2025:08:12:46 +0000] "POST /api/v1/database/ HTTP/1.1" 500 469 "http://localhost:8088/databaseview/list/?pageIndex=0&sortColumn=changed_on_delta_humanized&sortOrder=desc" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
2025-01-08 08:12:52,777:DEBUG:superset.stats_logger:[stats_logger] (incr) health
127.0.0.1 - - [08/Jan/2025:08:12:52 +0000] "GET /health HTTP/1.1" 200 2 "-" "curl/7.88.1"

```

And this is the content of dremio.conf inside the docker instance:

```auto
dremio@52cbec2a46d8:/opt/dremio/conf$ cat dremio.conf
#
# Copyright (C) 2017-2019 Dremio Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

paths: {
  # the local path for dremio to store data.
  local: ${DREMIO_HOME}"/data"

  # the distributed path Dremio data including job results, downloads, uploads, etc
  #dist: "pdfs://"${paths.local}"/pdfs"
}

services: {
  coordinator.enabled: true,
  coordinator.master.enabled: true,
  executor.enabled: true,
  flight.use_session_service: true
  # coordinator.flight.enabled: true
}

```

Inside Dremio logs:

```auto
14:34:57.542 [dremio-general-1] INFO com.dremio.service.reflection.MaterializationCache - Materialization cache updated. Materializations: reused=0 expanded=0 errors=0. External: reused=0 expanded=0 errors=0
14:34:57.543 [dremio-general-1] DEBUG com.dremio.exec.planner.plancache.PlanCacheSynchronizer - Reconciling plan cache with any changes
14:34:57.546 [dremio-general-1] INFO com.dremio.exec.planner.plancache.PlanCacheSynchronizer - Cleaning up 0 datasets based on entries, 0 datasets based on goals. Total (union) processed datasets: 0
14:34:57.546 [dremio-general-1] INFO com.dremio.exec.planner.plancache.PlanCacheSynchronizer - Completed plan cache sync. Cache entries before 2. Cache entries after 2.
14:34:57.546 [dremio-general-1] INFO com.dremio.exec.planner.plancache.CacheRefresher - Materialization cache sync took 1 ms. Plan cache sync took 3 ms.
14:34:57.694 [profile-sender1] DEBUG com.dremio.exec.work.protector.ForemenWorkManager - About to send all planning profiles
14:34:57.792 [UserServer-1] ERROR com.dremio.exec.rpc.RpcExceptionHandler - Exception in RPC communication. Connection: /172.27.0.8:31010 <--> /172.27.0.4:46096 (user client). Closing connection.
io.netty.handler.codec.CorruptedFrameException: Expected to read a tag of 10 but actually received a value of 82. Happened after reading 0 message.
        at com.dremio.exec.rpc.MessageDecoder.checkTag(MessageDecoder.java:256)
        at com.dremio.exec.rpc.MessageDecoder.decodeMessage(MessageDecoder.java:150)
        at com.dremio.exec.rpc.MessageDecoder.decode(MessageDecoder.java:68)
        at com.dremio.sabot.rpc.user.UserProtobufLengthDecoder.decode(UserProtobufLengthDecoder.java:32)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at java.base/java.lang.Thread.run(Thread.java:829)

```

Hope it helps!

---

<div class="post-metadata">

**Author:** ![relio](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/relio/32/6217_2.png) [@relio](https://community.dremio.com/u/relio)\
**Post date:** [January 8, 2025, 2:40pm UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/6 "2025-01-08T14:40:24Z")

</div>

Thank you Balaji, I’ve update the post with three logs.

Hope it helps!

Bye AR

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [January 16, 2025, 8:05pm UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/7 "2025-01-16T20:05:45Z")

</div>

Thanks for the logs, will review and get back

---

<div class="post-metadata">

**Author:** ![bardan](https://avatars.discourse-cdn.com/v4/letter/b/c0e974/32.png) [@bardan](https://community.dremio.com/u/bardan)\
**Post date:** [January 22, 2025, 3:18pm UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/8 "2025-01-22T15:18:52Z")

</div>

@relio  
Have you tried this?

I had to place this in requirements-local.txt in superset.  
git+https://github.com/lenoyjacob/sqlalchemy\_dremio.git and then rebuild the image…

and for the port number of dremio, I used 32010.

This is in case of open source configuration that is… Hope this helps.

---

<div class="post-metadata">

**Author:** ![relio](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/relio/32/6217_2.png) [@relio](https://community.dremio.com/u/relio)\
**Post date:** [January 22, 2025, 9:01pm UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/9 "2025-01-22T21:01:06Z")

</div>

Sorry, where is located requirements-local.txt in superset project? I cannot find it.

TIA!

---

<div class="post-metadata">

**Author:** ![bardan](https://avatars.discourse-cdn.com/v4/letter/b/c0e974/32.png) [@bardan](https://community.dremio.com/u/bardan)\
**Post date:** [March 10, 2025, 2:38pm UTC](https://community.dremio.com/t/cannot-create-dremio-connection-from-superset-exception-in-rpc-communication/12636/10 "2025-03-10T14:38:35Z")

</div>

@relio Sorry for late reply. I have a docker environment. So, based on that. After you have cloned superset repo. There should be a folder called docker in the cloned folder. **requirements-local.txt** is located there.