Question about Dremio resource requirement with Kubernetes Deployment

1

Hi, recently I noticed that the document says that Dremio needs about 8 cores of CPU and 16GB of memeory for the master coordinator, 4 cores of CPU and 16 GB of memory for the executor

but from the dremio_v2 helm chart on Github dremio/dremio-cloud-tools, which I believe is a official helm chart that will also deploy Dremio on k8s cluster, has default resource requirement that set both the coordinator and executor to about 15 cores of CPU and 120+ GB of memory.

Why is there such a big requirement difference?

And why is there also such a big resource requirement difference between the standalone cluster deployment and kubernetes deployment?
Seems that the standalone cluster version needs 16 CPU and 120+ GB of memory per node to operate and meanwhile according to the document the k8s version needs ways less resources to deploy, like 8 core of CPU and 16 GB memory, why?

Thanks.

2

Hey @david4221, Welcome to the Dremio Community!

The documentation states that Dremio typically requires a minimum of 16 CPU cores and 128 GB RAM per node.

Of course Dremio can run on far less specs but this completely depends on the performance SLAs you have set for your query workloads. Dremio typically scales linearly and is typically memory-bound. So feel free to test at lower specs and see if your query workload (with the concurrency you need) is able to fit in memory, and is able to meet your SLAs.

3

@lenoyjacob Thanks for the fast reply! I’ll try to adjust the workload resource request and see if it meets my requirement.