How to get all Description of Support Key?

1

https://docs.dremio.com/advanced-administration/support-settings.html

image
image.png826×589 23.5 KB

I see some support key by this url,but only 7.I want to get all Support Key of Description.
Can you tell me where I can find it?

2

We don’t recommend using support keys since they may change at any time and can lead to undesired behavior - they should only be used when absolutely necessary, which is why we do not document most of them.

3

Hi Doron,

Is there any support key that forces the dremio to use maximum available CPU? when we run concurrent queries the CPU utilisation is saying at less than 25% while the query duration is increasing. want dremio to efficiently utilise CPU and bring down the runtime.

4

@vamsik Dremio by default uses 75% of the # of cores on the node for every query, so say if you have 10 queries, every query will use 75% of the number of cores so it does get busy as concurrency goes up. In your case how many queries are running at a given point of time?

5

Tanks for the response Balaji.

We are trying to benchmark one of our data warehouse workload against dremio V23.1 vs Trino 405, from our initial analysis we see runtimes from both are almost similar though we expected dremio to perform better.

We observed that Inefficient utilization of CPU by dremio could be one of the reason for not preforming better than Trino as later is efficiently utilizing it.

With concurrency most of the dremio runtime is at Threads waits on CPU though CPU is available at server level and utilization do not cross 30% at any point of time.

we are testing with concurrency 5.
On the other hand with same queries and concurrency Trino is utilising 98% CPU and completing in the same time as dremio.

We think dremio run times can reduce 1/3 rd at-least if the CPU is efficiently utilised.

Also is there a way to increase the no of threads to 90% from 75%?

Thanks
vamsi

6

@vamsik Sometimes too many threads can also cause sleep times, any chance you can share profiles to review?

7

It makes sense now Balaji, Thanks for your valuable input.
is there a way to control no of threads in dremio? I understand default is 75%.

Thanks
vamsi

8

@vamsik parallelism

Pre unlimited splits for scans is min(below 3)

  • number of splits in final physical transformation
  • 70% of number of cores can be controlled by planner.width.max_per_node_fraction
  • row count estimate (not actual) - thread for every 100K, can be controlled by planner.slice_target

Pre unlimited splits for other operators other than scans is min(below 2)

  • 70% of number of cores can be controlled by planner.width.max_per_node_fraction,
  • row count estimate (not actual) - thread for every 100K, can be controlled by planner.slice_target

Post unlimited splits for all operators

  • 75% of number of cores can be controlled by planner.width.max_per_node_fraction,
  • row count estimate (not actual) - thread for every 100K, can be controlled by planner.slice_target