I am trying to create a reflection on a table that has almost 52GB in size. My dremio cluster has 8 executors each having 6 cores and 30gb memory. I am getting the error
Query was cancelled because it exceeded the memory limits set by the administrator.
I tried increasing the executor count to 12 and still getting the same issue. I even tried increasing heap memory by providing the following in my dremio-env file
Even giving 20GB as heap memory gives me the same error. How can i make sure my reflection runs? How does dremio use heap memory and direct memory for query execution and reflection runs?