Caching - On Prem Implementation

1

Is caching (not reflections) for On Prem Installation of Dremio ? When i say caching I am referring to data that was queried previously from a remote location (S3) ? Will subsequent queries read from the local disks ?

Thanks
VRB

2

@vrb That is correct, this is called cloud cache, see documentation below. It is available for both datasets (PARQUET only) and reflections

https://docs.dremio.com/software/deployment/cloud-cache-config/

3

Thank you. Does the Key NUM_CACHE_HITS in profile indicate the cache hits ? if so what is the unit ? Number of page reads ?

4

@vrb Yes, indicates cache hits, will get back on the right number/unit for NUM_CACHE_HITS

5

Thank you. Also there is another field about NUM_CACHE_MISSES. Like to know what it indicates

6

@vrb Because we reading in chunks of 1MB while using async… we may read some bytes that are not needed… the difference would be small though, do you see a big difference? if so, can you please send us the profile?

NUM_CACHE_MISSES are reads back to the source like S3 as the block was not cached on local SSD (cloud cache)