I wanted to ask why we would not use AWS EFS and mount all executor nodes to that. This would enable them to share the cache. Using EBS seems to be problematic, if that node goes down you lose that cache (acceleration) until a new node can be provisioned. Also, does this limit where given sql statements can run or do the executor nodes communicate with each other to pull data sets across?
Hey @Russ_Wilson there are no issues with using EFS!
One concern would be pricing. S3 is the cheapest option but you pay a latency penalty. This is followed closely by EBS, which as you mentioned you have to deal with node failures. Then comes EFS which is significantly more expensive than both but also provides great performance and durability.