GC overhead limit exceeded

Dremio University
1

Hello Folks,

I am using dremio connector with mongodb and tableau client to access the data and do visualization. But I am recently facing this issue and getting error message like "ERROR ROOT - Dremio is exiting. There was insufficient heap memory to continue operating.
java.lang.OutOfMemoryError: GC overhead limit exceeded"

Could you help me on the insights of the issue and how can I fix this. This is also making my server CPU usage very high.

Thank you.

2

Hello @ansuman273,

This means your coordinator is running out of heap memory.

  1. Please share dremio-env and dremio.conf
  2. From your terminal, run ps -ef | grep Dremio and attach the output (this will show what memory settings the Dremio process is running with)
3

Hello Ben,

1: Please find the attached configuration files for your reference.
configuration.zip (1.8 KB)

2: We are using dremio on windows VM , please send me the equivalent command for ps -ef | grep Dremio on windows . I tried with Jps -l Dremio.

4

Hello @ansuman273,

Your VM is likely has too little memory for Dremio to plan and execute your queries (what are you memory settings for this VM?)

If you are experimenting with Dremio to get an idea of its capabilities, I highly suggest checking out the courses at Dremio University.

If you want to do more development / production workloads with Dremio, then I suggest scaling up your deployment to our minimum suggested hardware for coordinator and executors, and I would run those node roles (coordinator and executor) as different Dremio daemons on different host machines (your dremio.conf indicates that you are running both nodes roles as one Dremio daemon).