I’m running a 3 node cluster with 16 cores in each of the executor. Is there a support key settings somewhere or
maybe even in the dremio-env that allows me to configure my executors to use all 16 cores to its maximum potential? As I’m running queries, I noticed that via mpstat, I’m only utilizing 10-12% of each of the 16 cores. I feel like I’m not utilizing all of the cores available.
Dremio automatically takes into account all cores during job runs. There are ways to “force” more CPU usage but usually not necessary.
Maybe the jobs were so trivial they didn’t require the horsepower (like just regular queries)? Typically reflection jobs are resource intensive, maybe you can test one of those to see if it uses more? If you can share a query profile, we can also see more of what’s going on underneath the covers.
FYI - memory, on the other hand, is configurable & recommended to be set properly from the start - https://docs.dremio.com/deployment/system-requirements.html#best-practices