Need some clarity on the below concern:
How Dremio manages memory and hard disk space for executing each virtual data set. For eg: I have a huge dataset in my source and I created a virtual dataset and now I am querying and doing SQL operations on it multiple time to fetch the output. so, in this case, will it keep on consuming hard disk space to store the virtual data set result at any point of time. If it does consume how to check it should not go beyond the threshold of the space of the cluster in which Dremio is installed?
Do we have only the metadata stored for these VDS or data also gets stored in Dremio? If it happens which space it consumes and how can we track this space usage?