I was planning to create a little automation to kill ec2 machine nodes, when the cpu is higher than a specific threshold.
Before implementing this, I tried to kill one of the executors nodes, hoping that dremio would have spin up new ones. But this don’t happen, the engine stay in Starting, without provisioning a new node.
I was wondering what is the best way to achieve this. There seems not to be an API to call, to handle executors. Any hints?
This is a workaround to be able to use properly reflections.