Hi,
I’ve been trying to configure my dremio.conf file to use S3 distributed storage rather than local storage. No luck. I created a core-site.xml file with my AWS credentials and gave my S3 bucket appropriate permissions.
The issue I’m facing is that I get spilling errors on memory that prevent me from creating reflections or completing CTAS exports. Dremio keeps using my local Windows drive to store data despite my configurations.
Please find my dremio.conf attached.
Where and how do you have Dremio deployed?
Hi Kelly,
I have Dremio deployed on my laptop. It’s a Windows 10 machine with 16GB ram. I did a standard install of Dremio on Windows so it’s installed on my local hard drive (c:). Dremio launches on my local host ( http://localhost:9047). I hope this helps.
Thanks,
Arun
I don’t know if that config should work, but it’s not recommended. If you want to be using S3 for storage of your data reflections then you should have Dremio deployed on AWS instances.
