Building Wide Table - One or more nodes lost connectivity during query

I am building a wide table in dremio and encountered the below exception.

ExecutionSetupException: One or more nodes lost connectivity during query. Identified nodes were [dremio-executor-0.dremio-cluster-pod.dremio-v2-prod.svc.cluster.local:0].

My wide table consists of around 2300 columns.

I am using the OSS edition.

How can I find the server logs to see what caused the loss of connectivity? I saw my dremio executor pod in kubenetes got restarted though.

Attached is the job profile. (796.1 KB)

I resolved the issue by manually partitioning the columns into several tables.


Hey there, @Ken
I can see that you are having problem with your wide table in Dremio. Here’s what you can do:

  • Check Logs: Use the following command to see your Dremio pod logs in Kubernetes:
  • Job Profile: Open the job profile in Dremio’s web UI under “Jobs” to see where it failed.
  • Ensure Resources: Wide tables need lots of resources. Make sure your Dremio cluster has enough CPU and memory. You might need to scale your Kubernetes cluster.
  • Check Network: Ensure your network is stable and there are no interruptions between Dremio nodes.
  • Restart Cluster: Try restarting your Dremio cluster and ensure nodes are balanced. Scale down and then up in Kubernetes.
1 Like