Building Wide Table - One or more nodes lost connectivity during query

1

I am building a wide table in dremio and encountered the below exception.

ExecutionSetupException: One or more nodes lost connectivity during query. Identified nodes were [dremio-executor-0.dremio-cluster-pod.dremio-v2-prod.svc.cluster.local:0].

image
image1851Ă—884 108 KB

My wide table consists of around 2300 columns.

I am using the OSS edition.

How can I find the server logs to see what caused the loss of connectivity? I saw my dremio executor pod in kubenetes got restarted though.

Attached is the job profile.

7e0d89f6-6300-4b23-846e-6f384a636ae8.zip (796.1 KB)

2

I resolved the issue by manually partitioning the columns into several tables.

image

3

Hey there, @Ken
I can see that you are having problem with your wide table in Dremio. Here’s what you can do:

  • Check Logs: Use the following command to see your Dremio pod logs in Kubernetes:
  • Job Profile: Open the job profile in Dremio’s web UI under “Jobs” to see where it failed.
  • Ensure Resources: Wide tables need lots of resources. Make sure your Dremio cluster has enough CPU and memory. You might need to scale your Kubernetes cluster.
  • Check Network: Ensure your network is stable and there are no interruptions between Dremio nodes.
  • Restart Cluster: Try restarting your Dremio cluster and ensure nodes are balanced. Scale down and then up in Kubernetes.
1 Like