When I first installed Dremio I set it up for myself on my account on our cluster and created the dremio/data directory in my own home directory. Now I have moved dremio/data to a shared location on our scratch drive.
I have set up Dremio for other users to test out now and whenever they try to create a dataset it fails because it is still trying to write the results to the dremio/data/pdfs/results directory in my HOME directory and not the appropriate shared one on scratch.
I’ve added this line to my dremio.conf:
the local path for dremio to store data.
I’ve looked through dremio-env as well and the path it is trying to write to is not defined anywhere. I’m guessing this has something to do with me copying the dremio/data directory from my home to scratch. Is there a way, without me reinstalling everything, to fix this problem (i.e. does anyone know where Dremio is being told to write results to $HOME/dremio/data/pdfs/results rather than scratch or how to overcome this problem)?
Any insight would be greatly appreciated!
Thanks in advance