I deployed Dremio from the aws market place. However I like to change the instance type for the master and executors to use some of the reserved instances I own. I also want to deploy spot instances for some of my executors (for certain projects). How can I change these ? Is there a config file that I can edit ? Also I am fairly new to this, so is Dremio behind the scene doing cloud formation to deploy the executors ?
can anyone help me with this ? it will be great if someone can help
Hi @rajupillai,
Dremio does not yet support custom instance types and we also don’t currently have support for spot instances.
So what you want to do is not possible yet.
Hi there, is the support for custom & spot instances still in the list of features in development for a future Dremio release ?
Thanks
Hi @balaji.ramaswamy ,
By custom instance, I mean changing the engine node type (e.g. m5d.2xlarge) with custom values (not limited to the ones supplied in the dropdown list in the elastic engine configuration in admin UI).
About spot instances, it would be nice to be able to specify how many executor nodes (could be a percentage) created by the engine could be created as EC2 spot instances.
That way, non-critical jobs (e.g. for development or evaluation) could be run at a lower cost.
Is it also possible in AWS version to manage the elastic engine manually (i.e. not only in the admin UI) ?
Thank you
Both are excellent ideas and will provide feedback to product management, currently not available