Clarification on AWS edition allowed ec2 instance types

At the time of the creation of elastic engine nodes, the list of available instance types (AWS_INSTANCE_TYPE_OPTIONS) is different from what is shown on AWS marketplace.

These options were hard-coded in the UI in this file

I was wondering the reason for this and the kinds of issues I’m likely to run into if I were to choose an instance outside of AWS_INSTANCE_TYPE_OPTIONSbut allowed by marketplace listing

Hello @udaykrishna5,

The additional options allowed by the AWS Marketplace listing are shown because we specifically allow the coordinator to be sized to those instance types. For our executors, we made the decision to only allow the types shown to ensure the best user experience on AWS.

Sincerely,
@RyanTse

@RyanTse I was specifically searching for this topic, so I’d like to ask a follow-up question.

Is there any way we can choose a smaller version than the m5d.xlarge? While we need much of Dremio’s functionality, it will be a long time before we will have the throughput needs to utilize that big of a machine.

Is there any way to force a smaller executor? Our budget is quite tight…

Thanks in advance,
Dan

@danielfcolbert,

At this time, we do not support choosing a smaller size. I will pass this feedback on to product.

Sincerely,
@RyanTse

@danielfcolbert you can actually force dremio to use m5d.xlarge by passing that in the instance type when editing the Request Body of the API call dremio makes when it provisions a new cluster and changing instanceType key within awsProps

You’ll see a POST call being made to /apiv2/provision/cluster which can be edited on firefox as shown here

But I’m pretty sure Ryan and team won’t be able to guarantee stability so take that into consideration before making that gamble

@udaykrishna5 - thanks for the reply.
I have already done that - and that instance type is actually supported by the application.

I am talking about using an even smaller instance than that. And @RyanTse was kind enough to answer - even though it was different than the answer I wanted. :slight_smile:

Cheers,
Dan