My goal is to enable quick troubleshooting without having to SSH into the dremio instances on AWS. I am considering building a method to copy the dremio logs mentioned here https://docs.dremio.com/advanced-administration/log-files.html
Do I need to track logs from the executor nodes? What do they offer that the master coordinator does not log itself?
I plan to implement something similar to this AWS blog on exporting logs to S3 https://aws.amazon.com/blogs/mt/how-to-export-ec2-instance-execution-logs-to-an-s3-bucket-using-cloudwatch-logs-lambda-and-cloudformation/
I assume adding logging into a custom cloudformation template will add some maintenance burden. Are there any existing methods or best practices to use?