I’ve been trying to configure my dremio.conf file to use S3 distributed storage rather than local storage. No luck. I created a core-site.xml file with my AWS credentials and gave my S3 bucket appropriate permissions.
The issue I’m facing is that I get spilling errors on memory that prevent me from creating reflections or completing CTAS exports. Dremio keeps using my local Windows drive to store data despite my configurations.
Please find my dremio.conf attached.